TIBCO Clarity Feature Summary
It's a challenge to make use of all the data your business has access to. The quality of data impacts your decision-making and the accuracy of processes. Data coming from external sources such as SaaS applications or partners needs to be validated before it is used in systems.
TIBCO® Clarity makes it easy for business users to profile, validate, standardize, transform, deduplicate, address cleanse and visualize data so that trends can be identified and smart decisions can be made quickly. TIBCO® Clarity provides an easy-to-use Web environment and only requires an investment relative to the usage of the service.
Feature Summary Watch Videos
- Clarity on Cloud vs Clarity Enterprise Edition
TIBCO provides two versions of Clarity, which are Clarity on Cloud and Clarity Enterprise Edition:
- View Mode
TIBCO Clarity Enterprise Edition provides this view mode to allow multiple clients like vendor users directly access the projects by using the generated unique hash URLs. The master user will create and manage the datasets/projects, build up the operation list, and then share with their user clients. The user clients are only able to view the specific project, and to manipulate the project data.
- Multiple Data Sources and
TIBCO Clarity supports uploading raw data from disparate sources in wide variety of data formats:
- File formats: CSV, TSV, *SV, TXT, XLS, XLSX, JSON, XML, and the compressed formats (.zip, .gz, .bz2, .7z)
- Cloud storage: Box, Dropbox, Google Drive, Amazon S3
- Database: Oracle, Microsoft SQL Server, MySQL, PostgeSql and Amazon Redshift
- Data management software: TIBCO MDM, TIBCO ActiveSpaces, TIBCO Vault
- BI tools: TIBCO Spotfire (Synchronizing)
- Marketing tools: Salesforce, Marketo
- Web URL
- Still more to come, like mongoDB, DynamoDB, Microsoft Dynamics CRM, OData and so on
TIBCO Clarity provides the Mapping function that you can use to consolidate source data from multiple data sources into a unified dataset. Two mapping methods, auto mapping and manual mapping, are available.
- Data Consolidation and ETL
TIBCO Clarity can automatically run the ETL process. After importing the source data from any of the above mentioned sources, you can map and combine them into a single table, and then clean up the data in the table by applying various transferring and validating rules. At last, you can export the cleansed data to various formats.
TIBCO Clarity provides the Sampling function that you can use to select a portion of source data as the sample data, when the dataset is too large. Several sampling options are available, such as are based on the number of rows, and that are based on percentage, and so on.
TIBCO Clarity provides the Batch function that you can use to run batch jobs to apply the operations performed in any project of the dataset to the entire dataset.
TIBCO Clarity provides the Validation function that you can use to validate your project data according to the predefined data types or use-defined data types plus several constraints.
- Custom Data Types
TIBCO Clarity provides the function that you can use to define a set of customized data types. A custom data type consists of a basic data type and a constraint. The basic data types include String, Integer, Long, Double, Float, Boolean, Time, Date, and DateTime. TIBCO Clarity supports using regular expressions as constraints.
TIBCO Clarity provides the Undo-Redo function that you can use to revoke all the operations that you have performed and saved in a project. You can use this function to revert data to a previous status.
TIBCO Clarity provides the seamless integration with a powerful pattern-search engine (additional purchase is needed for Enterprise Edition) that you can use to find duplicate records. You can customize your match strategies by columns or by swappable columns or by columns against to a thesaurus table, and then use them to discover duplicate records from any incomplete or partially similar records, even from records in different languages. In addition, you can run de-duplication against the dataset or an external master table to correct the duplicate records. TIBCO Clarity supports uploading a data file and Clarity dataset to the TIBCO Patterns server as the master table or a thesaurus table.
- Email Address Validation
TIBCO Clarity provides the email validation function, which allows user to identify and verify whether these email addresses are valid or invalid in your dataset.
- Phone Number Validation
TIBCO Clarity provides the phone number validation function, which allows user to choose different vendors to identify and verify whether these phone numbers are valid or invalid in your system.
- Address Cleansing
TIBCO Clarity provides the Address Cleansing function that you can use to identify, correct, and standardize the address data in your dataset. Three different address cleansing vendors are provided here, which include TIBCO GeoAnalytics, Google Maps and ArcGIS. More new vendors can be available upon request.
TIBCO Clarity provides the Profiling function that you can use to analyze your data just by a simple click. You can use this function to do row analysis and column analysis, which collects statistics and information about your data. TIBCO Clarity can automatically detect the column type according to the profiling result.
TIBCO Clarity provides the Charting function that you can use to analyze your data by various charts, such as pie charts, line charts, and so on. You can apply different aggregation rules and grouping rules to adjust the chart.
- Built-in Functions
A dataset contains the entire source data that you want to clean up. You can select a portion of data or the entire data in the dataset to be included in a project. For each project, you can use the following operations to analyze and clean up the project data:
Faceting provides a different way of looking into your data and regrouping it. After faceting, only the rows corresponding to a given value are displayed. The following faceting methods are available:
- Global Faceting by Star
- Global Faceting by Flag
- Global Faceting by Validation
- Global Faceting by Empty Rows
- Column Based: Text Faceting
- Column Based: Custom Text Faceting
- Column Based: Numeric Faceting
- Column Based: Text Pattern Faceting
- Column Based: Timeline Faceting
- Column Based: Scatterplot Faceting
- Column Based: Customized Faceting
- Editing Rows
You can edit or remove rows according to the faceting result.
TIBCO Clarity provides the Search function that you can use to perform a case-sensitive text search and regular-expression search. You can do the search in the entire project or just in a specified column.
- Checking Dependency
TIBCO Clarity uses the key-value pair to check the column dependency.
- Record Aggregation
TIBCO Clarity provides the Record Aggregation function that you can use to perform computations into a record. That is, if an entity contains several records in your dataset, you can use this function to combine them and only show the most important metrics to represent this entity.
- Editing Columns
TIBCO Clarity provides a set of operations to edit columns in your project. Some of them are global, applied to all the columns; and some of them are local, only applied to a single column:
- Reorder columns
- Remove columns
- Merge multiple columns
- Covert empty string to null
- Trim the white space in the leading or trailing or both
- Splitting a column: splitting a column into several columns by condition and order
- Adding a column: adding a column based on the current column or based on fetching URLs
- Rename and move a column
- Format conversion on phone number: automatically correct and standardize the phone format according to the country information
- Format conversion on date time: automatically correct and standardize the date format based on user requirement
- Editing Cells
TIBCO Clarity provides various ways to update values for a specific cell or a set of cells at one time.
- Transforming Cells
TIBCO Clarity provides a set of methods that you can use to transform cell values in a column:
- Numeric Rounding Transform
- Numeric Ceiling Transform
- Numeric Flooring Transform
- Numeric logarithm Transform
- Trim White Spaces Transform
- Collapse Consecutive White Spaces Transform
- Convert Empty String to Null Transform
- Unescape HTML Entities Transform
- Change the Capitalization Transform
- Remove All the Cell Values
- Customized Transforms: define a regular expression to transform
- Splitting and Merging Cells
TIBCO Clarity provides the Splitting and Merging Cells function that you can use to split one cell into several rows when this cell contains multiple values delimited by a separator, and the other way around.
- Blanking and Filling Cells
TIBCO Clarity provides the Blanking Down function that you can use to clear the cells that contain the same value as a preceding cell, and the Filling Down function that you can use to fill empty cells with the value contained in a preceding column.
- Clustering Cells
TIBCO Clarity provides the Clustering Cells function that you can use to group different cell values that represent the same entity.
- Transforming Cells
- GREL Functions
TIBCO Clarity provides the following built-in GREL functions:
- GREL Control
- GREL Function
- GREL Function Arrays
- GREL Function Boolean
- GREL Function Date
- GREL Function HTML
- GREL Function Math
- GREL Function Strings
- File Reloading/Adding
TIBCO Clarity provides the File Reloading function that you can use to reload your source data when you want to synchronize the changes made to the source data. Or you can use the Adding function to add the file that contains the changed data to the dataset.
- Dataset and Project Clone
TIBCO Clarity provides the Clone function that you can use to clone projects or datasets. You can reuse the operation list of a project, and add more new operations based on the operation list.
- Exporting and Importing Dataset
TIBCO Clarity provides the Exporting and Importing Dataset function that you can use to share data, including the metadata in a dataset and projects, with other users.
- RESTful APIs
TIBCO Clarity provides a set of RESTful APIs that you can use to run TIBCO Clarity on other systems.
- Data Streaming
TIBCO Clarity leverages TIBCO ActiveSpaces as the internal queue to save the streaming data. TIBCO Clarity can continually process streaming data and write the result into a local file or a network file.
- Integrating with TIBCO ActiveMatrix BusinessWorks
TIBCO Clarity provides a set of RESTful APIs. You can call these APIS using TIBCO ActiveMatrix BusinessWorks to upload files to TIBCO Clarity, apply operations, and download result files from TIBCO Clarity.
- Synchronizing Data to TIBCO Spotfire
TIBCO Clarity supports synchronizing cleansed project data to TIBCO Spotfire just by a simple click.
- Integration with Odata Server
TIBCO Clarity provides a set of interface for the Odata integration, which allows users to import data from external Odata server into TIBCO Clarity and export data as Odata server.
- Integrating with Salesforce and Marketo
TIBCO Clarity provides this function to merge the object records of Salesforce or Marketo by using the de-duplication function. After removing the duplicates, TIBCO Clarity can synchronize data in two or more Salesforce and Marketo instances.