Metadata, Policies and Logs
by Hu Yoshida on Oct 2, 2006
I was happy to see Mark Lewis join the blogosphere last month. Mark has made a major impact on the direction that EMC has taken over the past 4 years. Coming from outside, Hewlett-Packard/Compaq, he is considered to be the new guard in EMC. He is writing his own blog, representing his own thoughts and not the corporate marketing spin. So I look forward to reading his blog to get his perspective on EMC, the company, their strategy, products and industry directions. Hopefully we can engage from time to time on topics of interest.
His recent blog on Network Intelligence, talks about the value of metadata. He defines data, "as the stuff we actually use" and metadata as everything else" including data that is captured in logs. I have a slightly different definition of metadata which does not include logs.
I believe data is content. Metadata, is descriptive data about the content. In addition to metadata, there are policies and logs. Policies are business rules about the use of the content. Logs capture events associated with the use of the data and of the storage it resides on. I make the distinction because meta data, policies, and logs have different roles to play. An application user can scan metadata to locate certain data, but he may not set policies or have access to the logs. An administrator can set policies and may see meta data, but may not have access to logs. An auditor should have access to logs to see that the policies have been implemented, but he can not change policy.
In our Active Archive Solution, Hitachi Content Archive Platform, we capture metadata from the content provider, associate it with policies that are set by an administrator through the HCAP management server. We also have a management log associated with our storage platform, that captures logins, settings, and information about operations which can be downloaded to a SYSLOG server.
Logs definitely play a role in compliance and forensics and is the basic tool for Security Event Management,SEM, as Mark points out. However, access to log should be limited to auditors and selected administrators. Today SEM products monitor host based or network based security point products, aggregating and correlating disparate events, to identify suspicious events and trends. Major systems vendors like Symantec, IBM, and CA as well as others like NetIQ, and e-Security provide such products today. Now with the availability of storage based logging, SEMs can expand their coverage to correlate storage and data events with host and network based events.
