In my last blog I discussed purging audit records from the Web Infrastructure Platform SharedServices database. The blog generated a fair bit of discussion around the SAS Visual Analytics auditing data gathering and archive process. So let’s take a step back and in this blog review how data collection for auditing works and where to look for logs to debug issues. I will do the same for the archive process in a follow up blog.

In this blog we will look under the covers rather than show you how to implement auditing. For a great summary of how to set all this up, see this YouTube video by my colleague Bobbie Wagoner.

When auditing is enabled, audit records are continuously generated when user activity occurs in the environment, and stored in the Web Infrastructure Platform service database. Two tables in the SharedServices database, the SAS_AUDIT, and SAS_AUDIT_ENTRY tables, are used to record activity that occurs in SAS web applications. By default, user logon and logoff events are recorded in these tables for all SAS Web Applications. When middle-tier auditing is enabled for Visual Analytics, additional user-activity is recorded to these tables and fed to Visual Analytics to support the Administrator overview report.

The SAS_AUDIT table contains records for each user interface action and the SAS_AUDIT_ENTRY contains more details about the action that occurred. In SAS_AUDIT, if we look at the first row below, the one with audit_id=12386, we can see that an action occurred for the object type 32 (object_type_id).

To determine what kind of object that is, we can view another SharedServices table the SAS_TYPE_OBJECT table. This table maps an object_type_id to a SAS metadata object. In the SAS_TYPE_OBJECT table we see that object 32 is a table.

Similarly the action code (action_type_id) in SAS_AUDIT is 36. Viewing the SAS_TYPE_ACTION table shows that action 36 is an Add.

Bringing this information together with the matching records in the SAS_AUDIT_ENTRY table (audit_id= 12386 shown below) reveals that the audited action is an add of the CUSTORDERS table to the LASR Analytic Server.

With Visual Analytics auditing enabled, data like the example above will accumulate in the SharedServices audit tables. You can check that data is being collected by viewing the tables in PgAdmin, as illustrated in this blog.

Enabling auditing also starts a process which extracts the data from the database and copies it to the Append directory of the EVDMLA library. This process uses the Visual DataBuilder web application and the SAS Pooled Workspace to perform the extraction.  To complete the implementation, auto-load must be enabled for the EDVMLA library. This document provides a good overview of how this complete process works.

The EVDMLA data library is used by SAS Environment Manager Service Architecture framework to auto-load data from environment manager auditing and monitoring activities to Visual Analytics. The Visual Analytics middle-tier auditing discussed in this blog is completely separate from EV Service Architecture Framework. You do not need to enable Service Architecture Framework to get it working, however it does share the same Visual Analytics auto-load location.

To debug any problems in the process, you can configure additional logging on the Visual Data Builder web application and the Pooled Workspace server. For SAS Visual Data Builder, add the following XML to the Lev1/Web/Common/LogConfig/SASVisualDataBuilder-log4j.xml and restart the Web Application Server.

<category additivity="false" name="com.sas.svcs.content.vdq.batch.audit">
<priority   value="INFO"/>
<appender-ref ref="SAS_CONSOLE"/>
<appender-ref ref="SAS_FILE"/>
</category>

The result will be that the extraction process writes a note to the log on the middle tier machine at Lev1/Web/Logs/SASServer12_1/SASVisualDataBuilder7.3.log.  You can see that it runs the extract every 15 minutes.

in the Pooled Workspace Server set the level of the App.Program logger to INFO in /Lev1/SASApp/PooledWorkspaceServer/logconfig.apm.xml (or logconfig.xml) and restart the Object Spawner.

<logger name="App.Program" immutability="false">
    <level value="INFO"/>
</logger>

The result will be that the extraction process writes a note to the log on the compute machine at /Lev1/SASApp/PooledWorkspaceServer/Logs/ showing the audit records being coped to the audit_visualanalytics table in the Append directory of the EVDMLA autoload location at Lev1/AppData/SASVisualAnalytics/ VisualAnalyticsAdministrator /AutoLoad/EVDMLA/Append.

The next time the autoload process runs, it notices that the audit data in the Append subdirectory is newer than the corresponding LASR table. The autoload process appends the new data to the existing LASR table. In addition, to ensure that data is available in memory after the server restarts, appended data is immediately written to a second location, the AUDIT_VISUALANALYTICS table in the autoload data directory.

The log for this process is in the logs directory for that autoload location /Lev1/AppData/SASVisualAnalytics/VisualAnalyticsAdministrator/AutoLoad/ EVDMLA/Logs/. Examining the log in that location, you will see the data being loaded into memory.

To summarize:

• Enabling auditing causes user activity to be recorded in the audit tables of the shared services database. You can check the table content with pgAdmin to see if this is working.
• Data is extracted from the database and written to the autoload location. Enable additional logging on the Visual Data Builder Web Application and the Pooled Workspace Server to debug any issues.
• When autoload is enabled for the EVDMLA library data is loaded into memory.

In the next blog we will review the archive process and look at some issues you may encounter and how to debug them.