If you are like many SAS Enterprise Guide users, you've amassed a large collection of project files (EGP files) that contain important content: programs, logs, notes, results, and more.
However, to most tools and processes, the EGP file is opaque. That is, you can't see what's inside of it unless you actually open it within SAS Enterprise Guide. And then you might have to poke around for a bit, opening various items until you find the one you're looking for.
What if you had a way to search your collection of EGP files, without opening them, to find all occurrences of any text that you specify? Would that rock your world?
Introducing...the EG-Project-Search-inator! (That's my name for it, as I've been watching a lot of Phineas and Ferb lately and so I'm inspired by Dr. Doofenshmirtz's naming conventions. With this, you can search every project file in the Tri-State area!)
The more-mundane-and-less-cool name is EGPSearch, and I built it as an example for my SAS Global Forum paper. It shows how you can use Microsoft .NET to automate SAS Enterprise Guide. But it also turned out to be a very handy tool.
>> DOWNLOAD link for the EGPSearch tool (SAS Enterprise Guide 4.3 only)
The download is a ZIP archive with the EXE and the Microsoft .NET source code that I used to build it.
UPDATE 25Jan2013: I've built a new version of this tool based on your comments and feedback. Use the version in this post to see how the automation API works, but if just want just the tool, try the updated version from this post.
It's simple to use. You place the EGPSearch43.exe anywhere on a machine that already has SAS Enterprise Guide 4.3. Then run EGPSearch43.exe to start it. Here's a screenshot of the user interface:
You specify a project file to search (as a fully qualified path name) or a collection of projects in a folder (using wildcards, such as "C:\\Projects\\*.egp"). Then specify the text that you want to search for.
The EG-Project-Search-inator will use the SAS Enterprise Guide automation API to open each EGP file and search for the text. The search will include all programs (including task-generated code), log output, notes, and process flow labels. The search is case-insensitive, so searching for "libname", "LIBNAME", or "libNAME" will find all of the same content. All of the results ("hits") will be displayed as a list in the main window. The log area at the bottom will show you which project files were opened and searched successfully.
With a tool like this, you can begin to answer all sorts of questions, such as:
- Where am I using a LIBNAME statement in my code? (Administrators would love to know!)
- I say that I love PROC SGPLOT, but how much do I really use it?
- What projects have ERROR lines in the logs? How about WARNING lines?
If you have burning questions that you want to pose to the Search-inator, download it today and give it a try.
Important notes about the EG-Project-Search-inator!
Before you download this tool, there are a few things you should know.
This tool is provided as an example, and isn't supported by SAS Technical Support. If you have questions or feedback about the tool (such as "Works great!" or "Curse you, Perry the Platypus!"), please tell me in the comments.
It works only with SAS Enterprise Guide 4.3. I'm working on a 5.1 version, but I'm not ready to release it yet. The 4.3 version might have some luck with project files from 4.1 or 4.2, but don't count on it. And because it uses the Version 4.3 automation APIs, you must have SAS Enterprise Guide 4.3 installed on your machine.
[UPDATE 25Jan2013: I've built a new version of this tool that supports v5.1 as well as some other features.]
You might see a bunch of files/folders that are created during a search, but that don't get cleaned up. As a by-product of searching a project file, some of the project internal files are copied to a temporary folder for searching, and then are removed when the search is complete. However, some users have seen these files land in the current working folder (which not always a TEMP folder), and then remain even after closing the tool.
Even when you close the tool, the EGPSearch43.EXE process might not end immediately. This is most likely to happen when there is an error trying to open one or more EGP files from a folder. This doesn't hurt anything, but you might find that one of your EGP files is "locked for read" if you try to open it in the full SAS Enterprise Guide interface. Simply use Windows Task Manager to end the EGPSearch43.EXE process, which will free the file.
You should install the most recent SAS Enterprise Guide 4.3 hotfixes, as there are fixes that may help the tool to behave better, and might avoid the issues I just described.
36 Comments
Hi Chris
This tool is very welcome indeed. Do you have plans to have this search capability just with the current project, so right from within SAS Enterprise Guide?
Bruno, great idea. I think it's possible, but it would have to use a different approach. The automation APIs don't work so well from within the EG environment, especially when you want to examine the currently opened project.
Two words... Love it!
One of the most useful pieces of software I have used for a while.
I do have a couple requests for the next release. Can you make a browse button rather than having to paste in the directory and can you hide the request for user login credentials exception that appears?
All of the custom tasks etc. that you have created are such massive timesavers. Here's hoping that they become part of the standard builds in future EG editions.
Thanks :)
David, thanks for the feedback.
Good requests. I'm a bit limited since this application is abusing the EG automation API a bit, which makes some assumptions about connectivity to servers (as you would need for running the project, but not simply for analyzing what's in the project). You can use the File menu to browse to a directory though...apologies if that wasn't obvious. You also use the File menu to save your results!
This will be really useful - I cant wait for the 5.1 version..
Thanks Bob. I'm working on it. EG 5.1 uses the latest version of the .NET framework, and a few other items are different, so I've got a few adjustments to make. Will post it as soon as I can.
Bob,
I've posted a 5.1 version that you can try. Here is the download link. Let me know how it goes.
Are you related to Dr. Heinz Doofenshmirtz?
Ah, it's you, Perry the Platypus!
Excellent job! Do you know how many times I have searched through my SAS code knowing that I wrote a particular piece of code, and not being able to find it? Only to find it at a later date embedded in an EG project. I'll tell you how many times; so many times that I almost made it a practice to stop writing "good code" in the EG project and instead, save the "good code" as a .sas program within the EG project. That way it was easier to search and find. This tool is frustration saver!
Here's my wish. I wish this tool could be optionally called from the command line to bypass the GUI, and accept options and parameters similar to findstr. The output could be exacly what you currently do. The example below would search through all of your .egp projects in the specified directory and its subdirectories looking for "Perry".
EGPSearch43.exe /s "Perry" c:\users\sascrh\documents\*.egp
Jerry, good suggestion. Would help to answer the question, "where's Perry?"
Very useful. Especially when metadata changes, and it necessary to update 100's of EG projects for library references.
Anything I can do to bribe... er.... incent you to get the 5.1 version out? Even better, how 'bout taking the EG team to lunch and ensuring this gets into 5.2.
And as long as I'm being greedy, is there a way to expose this to Windows 7 search?
David, all great suggestions. I'll work on the 5.1 version soon; I've got it going, just a few glitches I'm trying to work through.
The EG team does have a feature like this on their roadmap, but I'm not sure of the timeline.
Would love to see the Windows search integration as well. The trick is coming up with a nice lightweight version that satisfies the "contract" of the search protocols.
Love the tool and hope you wont only make improvements to the 5.1 version - we are stuck on 4.3 for a while yet.
Would love:
a subdirectory /s option
to be able to limit results of searches to code only (Type=SAS Program)
directory browse button with wildcards for file name as suggested by David Henderson
sort by each of the the results columns by clicking on the column
Great tool!
Ross, I've just posted a new version for 4.3 and 5.1, and it supports the "recursive" subfolder option. Baby steps. Try it out by downloading from here.
Hi Chris,
Love your EGPSearch Tool!!! Tried it out in EG 4.3 today.
I have some initial feedback if you are interested...
* love the ability to search in multiple Enterprise Guide projects by browsing to the folder and then selecting a project file and then changing its name to *.egp
* the wildcard and case insensitive style search is great too - handy if you can't remember the complete phrase you are wanting :-)
* love that it also searches for the string in the EG code task... I noticed that it doesn't find the string if it is an EG renamed node task though (I renamed an EG Summary Statistics task to Sigma and when I searched for sigma it wasn't found) Future feature request?
* excited that I could right mouse click on the returned item and open it in Enterprise Guide... Was hoping I could. I initially double-clicked on it to test if I could, was disappointed, then got excited you have this feature via RMC. Awesome!
* noticed that you are not able to search multiple EG projects (*.egp) and subfolders. Would be handy to do so...
* not sure how the subfolders option works... Thought it might search in subfolders relative to the opened .egp file but found it didn't. Can you explain what this option does and any requirements needed (if any) on the .egp file? Do you need to have the relative file path option set in the project properties for instance?
* would be handy to be able to change the sort order by clicking on the column name - future request? ;-)
* it's great that you can also save your search results!
Look forward in letting sasusers know about it. Thanks for sharing!
Cheers,
Michelle
Thanks for the feedback, Michelle. I enjoy your constructive comments wrapped up in such enthusiasm.
Some of the items you asked for are *supposed* to be working (like recursive folder searches), so I'll need to look into that. I'm sure it's some bug or another in my logic, and I'll see if I can fix it quickly. Thanks for being an "early adopter".
Great tool! Like many others, searching for 'neat' bits of code I've written before is a time consuming and frustrating exercise.
It would be really good if this was packaged with the EG install - at many sites, adding .exe files is strictly controlled, especially when working through Citrix environments.
Good suggestion - thanks for the feedback. I'll work with the R&D team to see what we can do.
can i use sas enterprise guide 4.3 to open the folder of enterprise guide 5.1?
thx~
No, EG 4.3 cannot open a 5.1 project. In general, earlier versions of Enterprise Guide cannot open project documents from later versions.
Chris,
Great! I wondering if there all the automation features are on Linux? Can we automate in, say, Linux Shell script?
Thanks,
Qing
Qing, you can't automate the EG application on Linux because EG is a Windows-only application. So to automate the behaviors of the EG application, you must use a Windows scripting technology.
Of course, you can automate SAS processes in general on Linux, which I do often. In fact, I have a set of SAS processes that I maintain in EG, publish to Linux (as SAS programs in a file folder), script using Bash, and then schedule via cron.
Great tool!
I'm having some problems searching in 9.1.3 projects though. I've copied the 9.1.3 projects from a 32-bit PC to a 64-bit PC (where EG 4.3 is installed). I get the "Failed to open xxx.egp. Reason: This project must be converted to the current file format before using automation at this version.
Any idea on how i can solve this? Migrating to 9.2 is not an option for various reasons...
Cheers,
Anders
It worked with the revised version. Once again, Great tool!;)
Pingback: The best of SAS blogs for 2012 - SAS Voices
Pingback: How to search your SAS Enterprise Guide projects (2013 version) - The SAS Dummy
Hello, Great job. Do you plan to add a feature search and replace?
It would be powerful to update egp file when you change paths.
Thanks,
François
Francois,
I'm glad that you like the tool. There is another tool (MigrationWizard.exe) that ships with SAS Enterprise Guide. You'll find it in the EG installation folder. That tool allows you to find/replace references to servers, libraries, and files within EG projects. It operates on the "project item" level, and doesn't get into the text of SAS programs. Information about how to use the tool is in MigrationWizard.chm, which accompanies the file in the EG folder.
Hello,
Thank you for your reply. I know and use MigrationWizard. But in our case we migrate from Unix to Windows. We have to modify all the path included into the EG projects. We are looking for a tool to looking for a path like "\home\sasuser\my_data" and replace it by "C:\ConfigSAS\User_data".We have thousand EG projects to change.
If I understand, there is no tool for it.
Thanks,
François
François, that's right. Unfortunately there is not a tool (beyond MigrationWizard) that does what you want.
Fantastic - was just thinking how can I search through all the EG's without having to open each one and found this. Instead of hours searching through all the projects it took seconds to see what needed to be changed. Thank you!
Jerry
Tried running the tool with EG 7.1 HF1 (7.100.0.2002) (64-BIT). Installed search tool in a CITRIX environment in the same directory as SEGUIDE and get the following error window:
Cannot locate SAS Enterprise Guide 7.1.1. Is SAS Enterprise Guide 7.1 installed? Click on OK and program ends.
Any suggestions?
Pingback: My favorite SAS tips and tools from 2012 - The SAS Dummy
Pingback: How to search your SAS Enterprise Guide projects - The SAS Dummy