To find exact duplicates, matching all string pairs is the simplest approach, but it is not a very efficient or sufficient technique. Using the MD5 or SHA-1 hash algorithms can get us a correct outcome with a faster speed, yet near-duplicates would still not be on the radar. Text similarity is useful for finding files that look alike. There are various approaches to this and each of them has its own way to define documents that are considered duplicates. Furthermore, the definition of duplicate documents has implications for the type of processing and the results produced. Below are some of the options. Using SAS Visual Text Analytics, you can customize and accomplish this task during your corpus analysis journey either with Python SWAT package or with PROC SQL in SAS.
Using SAS Viya in combination with open-source capabilities, we were able to develop an automated solution for logo detection that does not require any manual data labeling.
Attend this session during the SAS Explore event on Sept 27-29 or view the recording at your convenience. We will showcase the use of SAS Intelligent Decisioning, SAS Model Manager, and SAS Visual Analytics on the SAS Viya platform for a solution that helps mitigate inequitable credit decisions.
SAS' Gunce Walton introduces to you a new scoring capability, how it utilizes Deep Neural Networks (DNNs) and shares use cases with PROC DEEPCAUSAL.
The IEEE Visual Analytics Science and Technology (VAST) Challenge provides a great opportunity to validate our software against real-world scenarios using complex data sets. Not only do we learn from these projects, but we also send feedback to our development teams to further improve product capabilities for customers.