Security event management, and the related SIM and SIEM, are computer security disciplines that use data inspection tools to centralize the storage and interpretation of logs or events generated by other software running on a network.
Overview
The acronyms SEM, SIM and SIEM have sometimes been used interchangeably, but generally refer to the different primary focus of products:
Log management: Focus on simple collection and storage of log messages and audit trails
Security information management : Long-term storage as well as analysis and reporting of log data.
Security event manager : Real-time monitoring, correlation of events, notifications and console views.
Security information and event management : Combines SIM and SEM and provides real-time analysis of security alerts generated by network hardware and applications.
In practice many products in this area will have a mix of these functions, so there will often be some overlap – and many commercial vendors also promote their own terminology.
Event logs
Many systems and applications which run on a computer network generate events which are kept in event logs. These logs are essentially lists of activities that occurred, with records of new events being appended to the end of the logs as they occur. Protocols, such as syslog and SNMP, can be used to transport these events, as they occur, to logging software that is not on the same host on which the events are generated. The better SEMs provide a flexible array of supported communication protocols to allow for the broadest range of event collection. It is beneficial to send all events to a centralized SEM system for the following reasons:
Access to all logs can be provided through a consistent central interface.
The SEM can provide secure, forensically sound storage and archival of event logs.
Powerful reporting tools can be run on the SEM to mine the logs for useful information.
Events can be parsed as they hit the SEM for significance, and alerts and notifications can be immediately sent out to interested parties as warranted.
Related events which occur on multiple systems can be detected which would be very difficult to detect if each system had a separate log.
Events which are sent from a system to a SEM remain on the SEM even if the sending system fails or the logs on it are accidentally or intentionally erased.
Security analysis
Although centralised logging has existed for long time, SEMs are a relatively new idea, pioneered in 1999 by a small company called E-Security, and are still evolving rapidly. The key feature of a Security Event Management tool is the ability to analyse the collected logs to highlight events or behaviors of interest, for example an Administrator or Super User logon, outside of normal business hours. This may include attaching contextual information, such as host information, identity information, and so forth. This contextual information can be leveraged to provide better correlation and reporting capabilities and is often referred to as Meta-data. Products may also integrate with external remediation, ticketing, and workflow tools to assist with the process of incident resolution. The better SEMs will provide a flexible, extensible set of integration capabilities to ensure that the SEM will work with most customer environments.
Regulatory requirements
SEMs are often sold to help satisfy U.S. regulatory requirements such as those of Sarbanes-Oxley, PCI-DSS, GLBA.
Standardization
One of the major problems in the SEM space is the difficulty in consistently analyzingevent data. Every vendor, and indeed in many cases different products by one vendor, uses a different proprietary event data format and delivery method. Even in cases where a "standard" is used for some part of the chain, like Syslog, the standards don't typically contain enough guidance to assist developers in how to generate events, administrators in how to gather them correctly and reliably, and consumers to analyze them effectively. As an attempt to combat this problem, a couple parallel standardization efforts are underway. First, The Open Group is updating their circa 1997 XDAS standard, which never made it past draft status. This new effort, dubbed XDAS v2, will attempt to formalize an event format including which data should be included in events and how it should be expressed. The XDAS v2 standard will not include event delivery standards but other standards in development by the Distributed Management Task Force may provide a wrapper. In addition, MITRE developed efforts to unify event reporting with the which was somewhat broader in scope as it attempted to define an event structure as well as delivery methods. The project, however, ran out of funding in 2014.
