Quantcast
Channel: Fortinet GURU
Viewing all 2380 articles
Browse latest View live

FortiSIEM What’s New in Release 4.4.1

$
0
0

What’s New in Release 4.4.1

Features

Windows Agent

Currently AccelOps collects Windows logs and performance metrics using WMI and SNMP, or via third-party agents such as Correlog and Snare. Pulling logs using WMI is expensive and difficult to maintain for high volume logging scenarios. Starting with this release, you can deploy

AccelOps agents to replace most of the above functionalities. AccelOps Windows agents can be purchased in two forms: Basic and Advanced. Basic agents collect Security/System/Application logs, IIS/DNS/DHCP logs, and custom log files. Advanced agents can additionally collect installed software changes, registry changes, file changes for file integrity monitoring,  and specific WMI and Powershell command outputs. Windows agents can be configured via AccelOps Windows Agent Manager using configuration templates. Windows Agent Manager communicates to the AccelOps Supervisor node for licensing/registration, and sends events to Collector or Supervisor nodes in compressed and encrypted form. AccelOps recommends that basic discovery and performance monitoring be carried out via SNMP/WMI, but the log pulling be performed via the agents. See Windows Agent Configuration for more information. Windows configuration manager is available on a separate license, contact sales@accelops.com for more information.

Beaconing

The Beaconing service transmits health and usage information about your AccelOps deployment to an AccelOps Cloud. Beaconing can be basic or advanced. Under basic beaconing, information transmitted includes the health of your AccelOps virtual appliances, CMDB device types, event parsing errors, performance monitoring job health, incident names, and summary information about the configuration of your deployment. Advanced beaconing includes system logs. Note that no specific host name, IP address or user information information is transmitted except the IP address of AccelOps virtual appliance themselves. This transmitted information is used exclusively by AccelOps support for forensic analysis of your system, and is never shared with anyone else. The basic Beaconing service is included as a standard feature in all 4.4+ versions of AccelOps, while a more advanced version can be purchased to provide additional log-based support services. The basic version is turned on by default but you can opt out at any time. See Using Beaconing to Communicate with AccelOps Support for more information.

External Threat Feed Integration Framework for Blocked Domains, Blocked IPs, Malware Hashes and Anonymity Networks

Before release 4.4, AccelOps already integrated with external threat intelligence feeds (such as Torproject.org, MalwareDomainList.com, ZeusTracker, EmergingThreats.net ) to populate blocked domains, blocked IPs, malware hashes and anonymity networks. However, the available integrations were mostly with free websites. Starting this release, user can integrate with their own paid content such as the Threat Stream OPTIC threat intelligence platform and others. A java based API is provided that enables you to integrate with any threat feed. If the threat feed is a website and the data is in the form of a comma separated file (csv) file format, then the integration can be accomplished from the AccelOps GUI itself by simply defining the column mappings and the separator. In all other cases, you will need to write Java classes based on examples provided with AccelOps 4.4. See the topics Custom Malware Domain Threat Feed, Custom Malware IP Threat Feed, Setting Up an External Data Source for Anonymity Networks and Custom Malware Hash Threat Feed for more information.

Integration Framework for External CMDB and Workflow Systems

This framework enables you to integrate AccelOps CMDB and incidents with external systems. Specifically, device information and new device attributes from an external CMDB, such as BMC Atrium, can be imported into the AccelOps CMDB. AccelOps CMDB data can also be

programmatically synched to an external CMDB, such as ServiceNow. AccelOps incidents can be pushed to a external workflow system, such as ServiceNow and ConnectWise – this integration is two-way, as changes in the ticket state in an external CMDB can be reflected back in the corresponding AccelOps incident. The integrations are built on a Java based API. While industry leading platforms such as ServiceNow and ConnectWise are already integrated out of the box, integrations with other CMDBs and workflow systems can be developed using the API. See the topics under Integrating with External CMDB and Helpdesk Systems for more information.

Data Update Service

AccelOps provides built in extensive device support in terms of device discovery, performance monitoring, log parsing, rules and reports. However until now, users had to wait for a formal product release, for example 4.4.2, to get new device support and existing device support extensions such as parser fixes, rule and report extensions. Starting with this release, customers can get device support enhancements, for example 4.4.1.101, via a data update service, in between formal AccelOps releases. As AccelOps continually adds support for more devices, by subscribing to this service, you can receive updated device support as it becomes available, instead of having to wait for a formal release. See the topics under Data Update Subscription Service for more information, and contact sales@accelops.com to purchase a subscription.

AccelOps User Management

This release enables AccelOps administrators to see all the currently logged on and locked out AccelOps users.  Users can be forcibly logged off from the system. Locked users can also be unlocked. Administrators can also see ongoing queries, the user who started the queries, and stop long running queries if needed.

User Interface and Navigation Enhancements

This release includes a number of enhancements to improve the user interface navigation and dashboards. Dashboard charts have now a flat look. The layout changed from column layout to cell layout where smaller charts can be combined with bigger charts on different rows. Cell size can be adjusted by the user on a widget by widget basis. The report selector has been redesigned. Single line chart now has a Gauge display in addition to text. Line charts can be stacked for better visual clarity.  The Table view and combo view now allows user to set colors based on displayed metrics. See the topics under Dashboard Overview for more information.

Revised Product Documentation and Customer Support Portal

The AccelOps product documentation wiki, as well as the customer support knowledge base and community forums, have been completely re-organized and revised for this release to improve the discoverability and usability of information. We welcome your feedback and suggestions for future development at infodev@accelops.com.

Enhancements

  1. Ability to monitor asymmetric network link utilization where send and receive link speeds are unequal
  2. Ability to exclude shared account names from Identity and location calculations
  3. Collector tunnel plugin launch should use super host name from browser to handle NAT deployment
  4. AO-SP: Every organization can have their own “My Home” country definition
  5. Ability to run a query with specific values from Dashboard Charts
  6. Ability to use Incident Category in Rule definition for filtering incidents for user defined rules
  7. Ability to query location name using Analytics framework
  8. Ability to choose a time period in Historical Search by dragging mouse over the time axis

Device Support

Device Access Protocols Used For
Cisco Meraki Cloud Controller, Cisco Meraki Firewalls, Router/Switches and Wireless Access Points SNMP Discovery and Performance

Monitoring

Syslog Security Event Management and

Log Analysis

SNMP Trap Availability Monitoring
Avaya Communication Manager SNMP Discovery and Performance

Monitoring

CDR files pushed to AccelOps via FTP or SCP Call record analysis
Windows Active Directory – health analysis by running dcdiag and repadmin/replsummary commands Remote command execution via

Winexe

Availability and Performance

Monitoring

Windows HyperV Monitoring Remote powershell via Winexe Availability and Performance

Monitoring

Dell Compellent Storage SNMP Discovery and Performance

Monitoring

Bit9 Security Platform Syslog Security Event Management and

Log Analysis

SourceFire NetworkAMP log analysis via syslog Syslog Security Event Management and

Log Analysis

Dell NSeries Router / Switch SNMP, Discovery, Performance

Monitoring

SSH Configuration change monitoring
HP Value Series Switches (19xx Series) and HP 3Com Switches (29xx Series) SNMP Discovery and Performance

Monitoring

SSH Configuration change monitoring

 

Bug Fixes

Edit Document

Bug Severity Module Description
5423 enhancement App Server Provide ability to tune event and per

Supervisor node

12646 major App Server Calendar view of incidents: actual # of
13424 minor App Server Collector tunnel plugin launch should u
13099 normal App Server (AO-SP) Every organization needs it o
11137 normal GUI On Analytics > Rule tab, it sometimes
11416 normal GUI User is not able to edit device under su
12042 normal GUI Drill down from Biz service dashboard
12833 enhancement GUI Can not delete Biz Service from CMDB
12955 normal GUI After editing a newly created user grou
13173 major GUI Identity and location exported PDF con
11350 normal GUI Sometimes the raw event log is empty
9285 major GUI Incidents triggered by user defined rule
       
10593 enhancement GUI Loading Analytics > Historical > Struc
11050 normal GUI A view only user should not be able to
11054 normal GUI If you only keep the Admin tab and hid
12169 normal GUI Quick Info > “Go to Identity” can’t find
12203 major GUI Deleting collector causes problems wh
12285 normal GUI Ticket belonging to an organization w
12539 normal GUI When you copy a search result to a new
12752 enhancement GUI Historical search prior time range menu
12783 normal GUI The Device Time attribute is not prope
12924 normal GUI Creating event dropping rule for an org
12961 normal GUI Custom Performance monitoring: delet
13665 normal GUI Enforce RBAC control on user tab – an
13673, 13625 normal GUI Chinese characters in UI when locale is
12232 normal GUI When user switches to an Organization
12241 normal GUI Important processes defined in Super/g
12246 normal GUI System defined device type will be ove
12274 normal GUI ON DNS Synthetic Transaction Monito
12346 normal GUI Cannot change port value on a newly c
12457 normal GUI Duplicated credential causes JDBC cus
12504 normal GUI An organization user can see Super Glo
12547 normal GUI Restrict customers from adding Organi
12708 minor GUI Need to (re)set to correct default port if
12774 normal GUI Parser XML editor: If search strings co
12802 normal GUI On Firefox browser, email subject does
12902 normal GUI User cannot delete an organization if u
12962 normal GUI Allow more than 255 characters in Reg
12973 normal GUI Restrict user from adding more than 16
13354 normal GUI Cannot delete authentication profiles fo
9973 enhancement GUI Allow user to bulk delete any CMDB g
10044 enhancement GUI Allow to display “latest” vulnerability a
11768 normal GUI CMDB > Applications > Running On t
12001 normal GUI Cloning and Moving CMDB Items resu
12140 minor GUI Should validate email address format w
12347 normal GUI Impact org shows in maintenance colum
12420 enhancement GUI Duplicate Components section in CMD
12434 normal GUI Can create duplicate biz service name i
12534 minor GUI Can not add / edit the description for an
12548 minor GUI Device Maintenance Takes Dates that a
12851 normal GUI CMDB Device Custom Property Thres
12870 normal GUI Allow CMDB Reports to be emailed in
12890 minor GUI The group name does not show when u
13552 normal GUI Drill down does not work for some of w
       

i

 

13681 enhancement GUI Add Location in the CMDB Search dro
2437 normal GUI For hosts, system uptime is calculated
6482 minor GUI Report sort order does not affect to wid
12085 enhancement GUI Extend Dashboard widget extend time
12381 normal GUI Invalid IP addresses with spaces can be
12517 normal GUI App Health page empty for EMC CLA
12724 normal GUI The sort function is lost in business ser
12876 normal GUI Duplicate “Free Array Storage” on Cla
13253 normal GUI Single Line widget on a dashboard doe
13639 normal GUI Dashboard Drill Down from Magnifyin
9610 normal GUI If any report is run with the “Run Late email all show Organization “Global”.
10314 normal GUI Reports with expressions in display col
11544 normal GUI When values are less than 1, heat maps
11804 normal GUI Provide an option to not have charts in
12223 normal GUI Date format in PDF is US date formate
12446 normal GUI Historical Search: Once stopped a quer
12764 minor GUI Schedule report date format should be
12775 enhancement GUI Need to shorten key info in incident vis
4320 enhancement GUI System-defined rule exceptions work f
12276 normal GUI New button is grey in Analytics > Rule
12362 minor GUI The drop down box of subpattern is too
12926 normal GUI Two rules (“Multiple Logon Failures: show “Triggered Event Count” inciden
13383 normal GUI Can’t see email template names in Ema
12454 normal GUI In CMDB -> Devices -> Topo (upper r
13288 normal GUI The incident count is wrong on Inciden
12528 normal GUI PDF export of Event Pulling errors doe
10285 normal GUI Add ability to mail::CC with Email No
13192   GUI In CMDB tab, a device should be filter
10531 normal Data Frequent SVN error – Could not create
10645 major Data InfoBlox NiOS SNMP based discovery
12395 normal Data Palo Alto Firewall: the event PAN-OS-
13600 normal Data Enhance IronPort web parser to cover d
13622 normal Data Sonicwall wlan logs from firewall not p
13667 normal Data Add retry for creating folder in phData
13683 enhancement Data Add Guaranteed eps to these events
13684 enhancement Data Add vmware datastore utilization rules
12411 normal Data Rule “Critical APC Trap” cannot be au
13179 normal Data Uncommon DNS Query Rule triggers u
9654 normal Data Some WinOSWmi Spanish events not
11864 enhancement Data Security Descriptor Field need to be pa
11930 normal Data Certain IOS events not parsed – IOS-E

 

      IOS-LAPP_ON_MSGS-LAPP_ON_

IOS-SWITCH_QOS_TB-TRUST_DEV

11993 enhancement Data Fortigate wireless AP events needs to b
12445 normal Data Incorrect test events for SyslogNG pars
13004 normal Data Need to resolve host name parsed from
13064 normal Data Sourcefire NetworkAMP events not pa
13338 normal Data Windows WMI and Snare parsers have
13341 normal Data Brocade SAN Switch events parsed to
13345 enhancement Data Windows System event types need to i
13390 normal Data Parsing error when [ in attr value in ph
13610 normal Discovery Special character “&” in host name cau
7726 major Identity Need to differentiate between domain u
12267 normal Parser Allow Netflow flows to be dropped lik
13612 normal Parser WMI events ‘Reporting IP’ not parsed c
13743 normal Parser PH_DEV_MON events have incorrect
12985 enhancement Parser Extend the Sender IP choice in Event F
11788 enhancement Performance Monitoring Pre-define some ssh/telnet/winexe jobs
12970 normal Performance Monitoring AO still pulls custom perf events after
13355 normal Performance Monitoring Oracle Acme Packet Controller Session
13611 normal Performance Monitoring Sonicwall interface not monitored corr
13619 normal Performance Monitoring Arista interface does not include link e
13629 enhancement Performance Monitoring Monitor load average for linux machin
13770 normal Performance Monitoring InfoBlox DHCP monitoring memory le
13640 normal Performance Monitoring VMware Cluster Consumed Memory v
11684 major Query / Report Query worker continues to perform sto
11847 normal Query / Report Query may not finish when event cand
10300 enhancement Query / Report Exported query results on super global
12747 enhancement Query / Report Allow customers to report on “Passwor
12884 normal Query / Report Exclude the event ASA-Update-Conn f
12919 normal Query / Report Exported Dynamic watchlists show inc
13439 normal Query / Report (AO-SP) The event PH_DEV_MON_ set to 1 – so network performance effic
12886 normal Rule Add reason for dropping events in PH poorly defined rules.
12913 normal Rule In rule synch error window, when you
10386 enhancement Rule When running Test Rule do not create
13609 normal Rule Network efficiency calculation is incor
10235 enhancement System Allow user to specify Super or Worker
10377 major System Fix the following vulnerabilities – CVE
10566 major System Fix the following vulnerabilities – CV

Cipher

10596 major System SVN password in EC2 build gets reset
       
11649 major System Failure to mount NFS on worker does n
12831 normal System Force AccelOps images to always mou
13008 minor System Disable SSLv3 and RC4 cypher by def
13690 normal System Installation script should ask the user t

in CMDB

Caveats / Open Issues

 

Bug

Id

Issue Workaround
6940 Rule/Query does not work with NULL non-string fields (e.g. Source IP). These entries are skipped. It works however with NULL string values (such as Host name). If Group By conditions have non-string fields, then make sure that those fields are parsed in events.
8867 LAST and FIRST operators in rule group event constraints causes Rule Worker modules to crash Avoid using LAST and FIRST operators in Rule group event constraints
11036 PctChange operator in rule group event constraints causes Rule Worker modules to crash Avoid using PctChange operator in

Rule group event constraints

11112 COUNT DISTINCT operations are expensive for anomaly rules Avoid using COUNT DISTINCT in anomaly rules
12900 Advanced HTTP STM via Selenium plugin does not work for some webpages – root cause is that AccelOps uses python export which does not support the full functionality of the browser plugin. Need to use java export instead of python export. None – use STM on simpler webpages.
13744 Empty strings in synched report results should be exported to Report Server as NULL instead of empty strings. Within Tableau, CAST conversion  operations FAIL when an empty string is encountered, but do not when a NULL is there. None

FortiSIEM What’s New in Release 4.3.3

$
0
0

What’s New in Release 4.3.3

AccelOps release 4.3.3 is focused on bug fixes and enhancements.

Bug ID Severity Component Description
13182 major Performance

Monitoring

Performance Monitoring jobs fail when devices discovery credentials are specified in sub-net notation
12604 major Event

Packager

Potential event loss if Supervisor node is down for extended amount of time
13010 major GUI Domain field is missing for manually added users for LDAP authentication
13098 major Rule Excessive Incident Drop Logging may cause parser module to consume high CPU
13020 normal App Server The ‘&’ character in a Rule name causes App Server out of memory error
13028 normal App Server When you discover with VM SDK first and then SNMP/WMI followed by consecutive VM SDK discoveries (e.g. VM SDK->SNMP/WMI->VM SDK -> VMSDK), then discovered information via other means (e.g.

SNMP/WMI) may be incorrectly deleted from CMDB

12953 normal App Server Clear Condition Attribute does not show up without saving the rule
13255 normal App Server Exception thrown during App Server start up caused by SystemConfigManager not found
13011 normal App Server Device maintenance does not work if End Date not set
12994 normal App Server Cannot query user-defined Watch list entries in Rules and Reports
13063 normal App Server Fix XSS vulnerability with Collector registration process
12939 normal App Server If there is an error in delivering a scheduled report, the report name is not captured in

PH_REPORT_ACTION_STATUS event

10302 normal App Server Incident count on Dashboard Calendar View page does not match count in Incident Tab
13027 normal Discovery Juniper SRX firewalls are sometimes discovered incorrectly as JunOS routers and therefore not put in the right CMDB group
13243 normal Discovery HP ProCurve SSH based configuration discovery fails for newer switches
13012 normal Discovery Cisco VoIP phones not discovered when phones do not have MAC address in SNMP walk (Call Manager

V10 and later)

12901 normal Discovery Discovery of Windows Server Host Names differ from 4.2.3 – FQDN do not have highest priority
13119 normal Discovery Discovering interfaces with /31 and /32 mask fails – traditionally these masks are not used in proper IP address definitions
13337 normal Discovery Use LLDP in addition to CDP for Layer2 Port mapping discovery – PCs connected to non-Cisco phones connecting to Cisco Access Switches
12891 normal Discovery Dell PowerConnect Switch configuration discovery via SSH fails for older switches
13190 normal GUI (AO-SP only) Editing the global exception for a rule would overwrite the org exception for the rule.
12921 normal GUI All Devices Dashboard Summary page does not populate when there is a special character (‘&’) in host name
12865 normal GUI Cannot drill down from widget dashboards by selecting a specific value in the charts – used to work in

3.7.6

12936 normal GUI Rules show incorrectly that Clear Condition is undefined, until clear condition is edited or viewed.
13233 normal GUI Report does not run when event attributes has %
13178 normal GUI CMDB Report for Active Rules does not work in Enterprise Edition
13315 normal GUI Dashboard error happens when user renames business service
13279 normal GUI Prev button does not work when searching for keyword in Custom Parser GUI
13221 normal GUI Edit Rule from Incident tab does not work correctly after Group By Name
12007 normal GUI Exceptions are not cloned when a rule is cloned
13122 normal Rule Rule does not fire if the DeviceToCMDBAttr function is used in Incident event type definition
13111 normal Parser Checkpoint certificate encode-decode inconsistent – an extra NULL termination character added to the SIC before encoding can cause SIC mismatch errors during decode and the LEA connection to be aborted

 

11253 normal Parser Possible bug in WatchGuardFirewallParser causes the parser to fail to extract any useful attributes from the log.
13249 normal Parser Avoid reverse DNS look up in syslog for host names that do not match host name criteria
12915 normal Performance

Monitoring

WINEXE does not work for Windows 2012 R2 – this is used for remotely communicating to Windows servers
12910 normal Performance

Monitoring

Custom winexe based performance monitoring: inconsistent behavior- sometimes test returns correct items, however sometimes it also does not return any data
12911 normal Performance

Monitoring

Custom winexe based performance monitoring: Unexpected “Variable <xxx> Not Found” error
13029 normal Performance

Monitoring

Use SNMP/dskTable (first choice) and SNMP/hrStorage (backup) to calculate disk space utilization and reserved space for Linux based systems. Create an event when SNMP/dskTable is not configured.
12845 normal Query Query tasks should be evenly distributed to all Worker nodes instead of assigning to first few Worker nodes
12968 normal Device

Support

False positive on Windows Audit Log Cleared rule caused by not considering Source Name in Windows event log parsing
13007 normal Upgrade Disable SSLv3 and RC4 cypher by default on ssl.conf while upgrading from 4.2.* to 4.4.*
13013 enhancement Device

Support

Add performance monitoring for FireEye MPS appliances
12980, 12979 enhancement Device

Support

Support Cisco Meraki Cloud Controller – discovery, syslog, SNMP trap
12647 enhancement Device

Support

Parse Cisco UCS syslog events
13057 enhancement Device

Support

Add new IPS signature definitions for Palo Alto FW/IDS
12925 enhancement Device

Support

Additional parsing for Sonicwall Firewall events: sent packets and receive packets
13023, 13154,

12946, 13285,

12929, 13001

enhancement Device

Support

Add more parsing for Windows security event logs
12895 enhancement Device

Support

Add event type Win-System-98 for detecting disk corruption
13312, 12933,

13271

enhancement Device

Support

Additional parsing for NetScaler login events
13113 enhancement Device

Support

Additional parsing for TrendMicro OfficeScan syslog
13047 enhancement Device

Support

Additional parsing for Cisco IOS syslog
12932 enhancement Device

Support

Additional parsing for Brocade network switches
13294 enhancement Device

Support

Additional parsing for CiscoNxOS syslog
13000 enhancement Device

Support

For JUNOS cpu monitoring, use 1.3.6.1.4.1.2636.3.1.13.1.21 (jnxOperating5MinLoadAvg) instead of

1.3.6.1.4.1.2636.3.1.13.1.8 (jnxOperatingCPU)

13014 enhancement Device

Support

Additional parsing Cisco Call Manager syslog events
12766 enhancement Device

Support

Add parser for Trend Micro’s Deep Security tool
13104 enhancement Device

Support

Support new format for Cisco IOS OSPF syslog message format
12989, 13103 enhancement Device

Support

Additional parsing for Cisco Ironport Mail appliances – “TCP_DENIED events
12930, 12931 enhancement Device

Support

Additional parsing for FortiGate events
13207 enhancement Device

Support

Discover Virtual Switch hardware information for JunOS

 

13120 enhancement Device

Support

Discover hardware information for JunOS via SNMP
13070 enhancement Device

Support

Parse Dell force 10 syslog
13042 enhancement App Server VA with Collectors: Duplicate devices when a device is discovered by Collector C1 while logs sent to different Collector C2
13043 enhancement App Server Incident Notification via XML/HTTP(S) – do not always require user name and password
13216 enhancement App Server Incident Notification via XML/HTTP(S) – Allow SOAP header and style sheet
13016 enhancement Rule Make the event delay threshold for Rule engine configurable
12996 enhancement Data Optimize “Heavy UDP Host Scan on Fixed Port” rule by excluding DNS traffic
13105 enhancement System Include nload and iotop as part of pre-packaged tools
12934 enhancement System Allow user to disable “Low AccelOps eventdb Disk Space” System Error Messages
10003 enhancement GUI Report event limit in exported or manual reports if the report result count is more than the supported upper limit
13234 enhancement GUI Add a drop down in CMDB for Collectors (in addition to Organizations) – this helps to identify devices associated with a collector
13002 enhancement GUI Add capability to search credential association by IP in GUI – should be able to search for an IP address within an address range or a subnet
13181 enhancement Performance

Monitoring

Provide a framework for computing host performance efficiency index based on Incident Triggers

 

 

FortiSIEM What’s New in Release 4.3.1

$
0
0

What’s New in Release 4.3.1

This release adds features and functionality in several areas.

AccelOps Visual Analytics

AccelOps Systems Features

New chart types for search result visualization

Visualization for profiled metrics and drill down from incidents

Performance and Availability Monitoring

Ability to specify per-device threshold in rules

Enhanced custom command output monitoring

Windows custom command output monitoring

Log Management and Security Incident Event Monitoring (SIEM)

Amazon Web Services CloudTrail monitoring

Box.com file monitoring

Okta Single Sign-On (SSO) integration via SAML 2.0

Vendor default password usage detection

Detect malware via file hash and user agent match

Detect communication via Anonymity Networks (Open Proxies and TOR nodes)

Device Support

Significant Enhancements

Migrate SVN to local disk

Trigger event query optimization

Device location import

Event dropping rule enhancement

CIDR format for specifying discovery ranges

Launch discovery from CMDB

IP Address management enhancements

Critical interface selection usability enhancement

CMDB Report extensions

Dynamic EPS Adjustment algorithm

Incident table and Identity/Location table partitioning

Paged control support for Microsoft Active Directory LDAP discovery

Events when device performance monitoring status changes

Enhanced custom parser development graphical user interface

Fixed Issues and Minor Enhancements

General GUI

Platform

Performance Monitoring / Event Pulling / Synthetic Transaction Monitoring (STM)

Rule / Query / Report Engine

Discovery

Device Support

Parsing  area

Data: System Rules/Reports

 

 

New Product

AccelOps Visual Analytics

This release enables AccelOps data to be visualized using Tableau Visual Analytics and Visual Analytics Desktop in conjunction with the AccelOps Report Server. Two kinds of AccelOps data can be visualized:

Data residing in Configuration Management Database (CMDB) e.g. Incidents, Device attributes

Any event database report result as long as it contains an aggregation condition e.g. GROUP BY

You can find full information in the Visual Analytics section of this wiki.

AccelOps Systems Features

New chart types for search result visualization

This release enables users to visualize query results within AccelOps using scatter plots, bubble charts, tree maps and heat maps. This complements existing visualization via pie charts, bar charts, trend charts, and geo maps. Scatter plots enable users to see correlations between any pair of calculated measures (e.g. CPU and memory utilization, Sent and received bytes etc). Bubble charts add a third dimension to scatter plots to reflect the size of the added dimension, e.g. in a scatter plot containing CPU and memory utilization as the two dimensions, the third dimension could be total sent and received bits/sec. The tree map is a hierarchical tree-structured visualization that is often used to analyze dominating components of multidimensional data e.g. IPS signatures, network traffic etc. Heat maps show the calculated measure for two dimensions using a color grade that helps users to understand severity. These charts are available both in Analytics and Dashboard areas.

This topic is discussed here.

Visualization for profiled metrics and drill down from incidents

AccelOps creates statistical baselines (profiles) for a large number of use cases. While earlier releases showed this information in tabular form, this data can now be visualized. For a specific dimension (such as host), up to four measures can be visualized on a trending hour-by-hour basis for weekdays and weekends as a multi-series column chart. In general, the profiles can be visualized as a scatter plot. From an incident indicating excessive deviation from statistical measures, it is now possible to drill down into the baseline with one click.

This topic is discussed here.

Performance and Availability Monitoring

Ability to specify per-device threshold in rules

AccelOps has rules that trigger when certain thresholds are crossed. When the thresholds have to be fine-tuned on a per-device basis, rule conditions become complex and difficult to manage. This release solves this issue. Instead of explicit threshold values in rules, the thresholds are now defined as custom properties in which the user can choose to override the global thresholds by redefining the thresholds locally for a certain set of devices. Instead of hard-coding thresholds, rules are now written using a function that returns the appropriate values – local values if one is defined, global values otherwise. This approach keeps the number of rules the same, but allows users to set thresholds for any number of devices. Thresholds can be a simple number (e.g. CPU utilization) or a map (e.g. interface utilization for each interface, disk utilization for one or more disks).

This topic is covered here (see the section: “Thresholds as Custom Properties”).

Enhanced custom command output monitoring

This release enhances the way custom performance monitor command outputs are parsed into events. Prior to release 4.3.1, command outputs are parsed as one line into one event. This does not include, for example, “show version” commands for Cisco IOS routers that can span multiple lines. Release 4.3.1 improves this situation – multiple lines can be parsed into one event.

This topic is covered here.

Windows custom command output monitoring

A Windows custom performance monitor can be used to bring PowerShell command outputs into AccelOps. Prior to release 4.3.1, command output was obtained via Telnet/SSH, but that is not natural for Windows, and the user had to install Cygwin Telnet/SSH in Windows systems. This release enhances the situation by using winexe client on AccelOps – any Windows shell commands, such as PowerShell, can be remotely run on Windows servers using WMI credentials. There is no need to install any software on Windows. Additionally, multi-line command outputs can be parsed into one event in AccelOps. This enhancement now enables customers to run PowerShell commands in Windows servers.

This topic is covered here.

Log Management and Security Incident Event Monitoring (SIEM)

Amazon Web Services CloudTrail monitoring

As more and more applications are deployed in the Cloud, monitoring user activity in the cloud is becoming increasingly important. For example, it is important to know when users are created, permissions are changed, virtual machines are spun up, network configurations are changed, or Virtual Private Clouds (VPCs) are created. This release enables AccelOps to efficiently collect, parse, report, and alert on Amazon Web Services activity via the AWS CloudTrail API.

This topic is discussed here.

Box.com file monitoring

Box.com is a cloud storage provider that is used by individuals as well as corporations to store and share files. This release enables AccelOps to monitor file activity within a Box.com account. AccelOps securely logs on to the Box.com account and monitors file creation, deletion, and modification activity within the account. More interestingly, for a specific file or all files in a folder, AccelOps can monitor file-sharing properties – is the file shared, is it password protected, is it preview/download enabled, and how many times was the file downloaded or viewed. If a particular file or directory contains confidential information, AccelOps can alert when any file in that directory was exposed to the outside or was viewed.

This topic is discussed here.

Okta Single Sign-On (SSO) integration via SAML 2.0

Oka is a cloud-based Single Sign-on (SSO) Service provider. This release enables AccelOps customers who are already authenticated in Okta to automatically log in to AccelOps without entering any credentials. AccelOps communicates via SAML 2.0 with Okta to verify user identity. In addition, AccelOps discovers all users defined in Okta (like Microsoft Active Directory) – the discovered users can be used in rule and report conditions and in notification policies. Finally, AccelOps collects Okta audit trails that can detect activity on the Okta web site such as account changes, logon activity, and other configuration changes.

This topic is discussed here.

Vendor default password usage detection

A common compliance requirement is to alert against the use of default vendor-defined credentials. This release enables AccelOps users to satisfy this requirement. AccelOps comes pre-built with a set of vendor and device specific default passwords. Users can add to this list.

Whenever a device discovery succeeds with a credential from this list, an alert triggers.

This topic is discussed here.

Detect malware via file hash and user agent match

This release comes with a set of built-in well-known malware user agents and malware file hash signatures. Users can also import their own lists from outside sources. Since malware is known to use non-standard http user agents, AccelOps alerts when it sees a malware user agent, regular expression-based match from web server or web-proxy logs. If AccelOps is configured for file integrity monitoring, then it can alert when it detects a malicious file hash match in a monitored directory.

Malware hash  is discussed here.

User agent is discussed here.

Detect communication via Anonymity Networks (Open Proxies and TOR nodes)

A compromised host or a user with malicious intent uses various techniques to hide their identity, with two common examples being proxies and TOR networks. This release comes with a set of built in well-known proxies and TOR networks. Users can also import their own lists from outside sources. Whenever AccelOps sees an IP address match from firewall logs or Netflow, an alert is created.

This topic is discussed here.

Device Support

Cisco VoIP infrastructure monitoring – see here

Cisco VoIP phone discovery from Cisco Call Manager via SNMP

Cisco Unity Connection – discovery and performance monitoring via SNMP

Cisco Presence Server – discovery and performance monitoring via SNMP

Cisco Contact Center – discovery and performance monitoring via SNMP

Cisco Tandeberg VCS – discovery and performance monitoring via SNMP

Cisco Telepresence MCU – discovery and performance monitoring via SNMP

More detailed performance monitoring of Cisco Call Manager – SIP Trunk Status, Gateway Status, H323 Device Status, Voice mail Server Status, CTI Device Status, Media Device Status

Parse 1000+ syslog messages from Cisco Call Manager and RTMT and create rules corresponding to RTMT Alerts

Oracle ACME Packet Controller – discovery and performance monitoring via SNMP

Brocade SAN Switch – discovery and performance monitoring via SNMP

Dell Force10 Switch – discovery and performance monitoring via SNMP – see here

Dell PowerConnect switches – discovery and performance monitoring via SNMP – see here

Nimble Storage – discovery and performance monitoring via SNMP

Cisco WAPX WLAN Controllers – discovery and performance monitoring via SNMP

MS SQL Server 2014 – discovery, performance monitoring, audit log  collection via SNMP, WMI, JDBC Oracle Audit log parsing via syslog

Wireless LAN Controller “module” on Fortinet firewalls. TrippLite Environmental Monitors

IBM WebSphere monitoring via HTTP(S) instead of JMX – see here

Arista switches and routers – discovery and performance monitoring via SNMP – see here VMware vShield – log parsing via syslog

Significant Enhancements

Migrate SVN to local disk

AccelOps uses SVN to store device configuration data and installed software information. Over time, this repository can grow and contain a very large number of files. Earlier releases hosted SVN over NFS, and network performance could become an issue over time. Since all accesses to SVN are via the Supervisor node, this release moves SVN to a Supervisor local disk on a separate logical drive. Fresh AccelOps 4.3.1 install automatically create a separate partition for storing SVN files. During AccelOps 4.3.1 upgrade process, a special pre-upgrade step is invoked to copy the SVN files over NFS to local disk. Actual upgrade does not begin unless existing SVN data has been copied over to the new disk – so the system continues to work during the pre-upgrade process.

Trigger event query optimization

Incidents are triggered by defined trigger events. When a user browses an incident in the graphical user interface, trigger events are shown, and incident notification emails can contain up to 10 trigger events. However, the AccelOps rule engine does not store raw events in memory, but only event identifiers, in an attempt to save memory. This means trigger events have to be retrieved from the event database by querying the event database. This query can be very expensive if the event is current, since the event may not have been indexed yet. This has been seen to create significant pressure on the AccelOps I/O system, especially if there is a sudden surge of incidents. This release addresses this issue by using an in-memory cache of raw messages for a short period of time.

Device location import

CMDB devices typically belong to private address spaces and their location is only known to the system administrators. There is now an easy way to input this information into AccelOps. Users can define locations by IP range or sub-net, and the location in CMDB will be instantly updated. The locations can be input manually via the graphical user interface, or imported from a file. In addition, devices can be searched by location in both the summary dashboard and CMDB.

This topic is discussed here.

Event dropping rule enhancement

Devices are often chatty and send all kinds of uninteresting logs to AccelOps. Since online storage is expensive, it is often necessary to be able to efficiently drop events before they are processed or stored. This release enhances event dropping rule framework by:

Including Source IP and Destination IP into the event dropping rule definition criteria

Allowing two different actions: drop completely, or store but do not trigger rules

Allowing the ability to automatically create drop rules from incidents in case the incident is a false positive, which is common in Network IPS event correlation scenarios

This topic is discussed here.

CIDR format for specifying discovery ranges

The test connectivity and discovery IP ranges can now be specified in CIDR notion as well.

Launch discovery from CMDB

Rediscovery can be directly launched from the CMDB page.

IP Address management enhancements

When allocating new addresses to hosts, it is often important to know the hosts that currently are assigned to addresses in a specific network segment. In prior releases, AccelOps discovered the network segments and showed only CMDB devices in that network segment. These do not include user devices such as laptops, workstations, mobile devices etc., since these devices do not necessarily belong in CMDB. Starting with this release, the Network Segment page also shows the hosts in the Identity and Location page belonging to the same network segment. Since AccelOps accurately learns all the IP addresses in a network via DHCP and IP ARP cache scan, administrators can correctly see every active host belonging to a specific network segment.

This topic is discussed here.

Critical interface selection usability enhancement

AccelOps allows users to mark interfaces as critical, and such interfaces are always monitored for utilization and up/down status. A common example is switch trunk ports, since a trunk port going down can cause a widespread network outage. Currently there is no easy way in AccelOps to select the trunk ports of all switches. Instead, the user has to traverse every switch and select trunk ports within that switch, which can be very tedious for a large network containing a large number of switches. This release provides a flattened view of the network interfaces so that a user can quickly select a large number of interfaces matching some search criteria. This enables administrators to mark all critical interfaces for a large network with only a few clicks.

This topic is discussed here.

CMDB Report extensions

CMDB Reports are extended to include

Successful Performance Monitor Reports

Failed Performance Monitor Reports

Identity and Location Report

Scheduled Report

Devices not updated in last N days

Dynamic EPS Adjustment algorithm

AccelOps has an algorithm to re-distribute unused EPS at a collector to other collectors seeing an event spike. The algorithm is now adjusted to have the following property: A collector is now always guaranteed to have the events-per-second specified as “Guaranteed EPS.” This EPS is never redistributed to other collectors. Only the excess EPS (defined as Overall EPS license minus the sum of all Guaranteed EPS) is redistributed on demand.

Incident table and Identity/Location table partitioning

In AccelOps CMDB, there are two tables that grow with time:

Incident table

Identity/Location table

The incident table grows as new incidents are created, while the Identity/Location table grows as new computers and users appear in the system or change location. As these tables grow, eventually the database may become full and read performance may suffer with corresponding growth in the table indices. In this release the following enhancements are made:

Incident Table Optimization:

The incident table is partitioned by month, so recent queries access the current month and result in fast returns During migration to 4.3.1 release:

Data for the last three months is migrated to the new tables (based on Last Seen Time field)

All ‘Active’ incidents are migrated

Older incidents are archived. Scripts are provided for customer to migrate older incidents into 4.3.1 CMDB. Scripts are provided to purge older incidents from 4.3.1 CMDB

Identity/Location Table Optimization:

The Identity/Location table is partitioned by month, so recent queries access the current month and result in fast returns During migration to 4.3.1 release:

Data for the last three months is migrated to the new tables (based on Last Seen Time field)

Older entries are archived. Scripts are provided for customer to migrate older identity/location entries into 4.3.1 CMDB. Scripts are provided to purge older identity/location entries from 4.3.1 CMDB

Paged control support for Microsoft Active Directory LDAP discovery

AccelOps discovers users in Microsoft Active Directory via LDAP protocol. By default, Microsoft LDAP search queries return up to 1000 entries per call (MaxPageSize limit – see Microsoft KB article). Earlier AccelOps releases required administrators to increase the MaxPageSize limit to a much higher number for user discoveries to work. This is generally inconvenient and may also cause resource issues on the server. This release enhances this situation. AccelOps LDAP discovery now uses the paged control version of the LDAP search API to fetch an arbitrarily large number of entries – 1000 at a time. Administrators are not required to increase the MaxPageSize limit beyond the default 1000.

Events when device performance monitoring status changes

AccelOps now generates audit events when the performance monitoring status of a job changes.

  1. User deleted a device or a collector:

<174>Nov 05 09:52:07

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2

,[srcIpAddr]=192.168.20.164,[phCustId]=1,[sessionId]=11178d2aeae08e

9c2babe2725fa1,[procName]=AppServer,[hostName]=HQ-A-Pxy-blueCoat,

[hostIpAddr]=172.16.0.141, [eventSeverity]=PHL_INFO,[customer]=Super,[jobStatusType]=UserDelet edDevice,[user]=admin, [phLogDetail]=Monitors on device were deleted due to device being deleted

  1. User disabled monitoring at a device level:

 

<174>Nov 05 09:53:58

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2

,[srcIpAddr]=192.168.20.164,[phCustId]=1,[sessionId]=11178d2aeae08e

9c2babe2725fa1,[procName]=AppServer,[hostName]=ACCELOPS-W2K3B4, [hostIpAddr]=192.168.64.124,[eventSeverity]=PHL_INFO,[custName]=Sup er,[jobStatusType]=UserDisabledDevice,[user]=admin, [phLogDetail]=Monitoring device, 192.168.64.124, is  disabled by user

  1. User enabled monitoring at a device level:

<174>Nov 05 09:54:38

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2

,[srcIpAddr]=192.168.20.164,[phCustId]=1,[sessionId]=11178d2aeae08e

9c2babe2725fa1,[procName]=AppServer,[hostName]=ACCELOPS-W2K3B4, [hostIpAddr]=192.168.64.124,[eventSeverity]=PHL_INFO,[custName]=Sup er,[jobStatusType]=UserEnabledDevice,[user]=admin,[phLogDetail]=Mon itoring device, 192.168.64.124, is enabled by user

  1. User disabled a specific job:

<174>Nov 05 09:55:17

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2 ,[phCustId]=1,[customer]=Super,[jobName]=System cpu usage,[srcIpAddr]=192.168.20.164,[appTransportProto]=SNMP,[sessionI d]=11178d2aeae08e9c2babe2725fa1,[procName]=AppServer,[hostIpAddr]=1 92.168.64.124,[hostName]=ACCELOPS-W2K3B4,

[eventSeverity]=PHL_INFO,[jobStatusType]=UserDisabledJob,[user]=adm in,[pullInteval]=180,[phLogDetail]=Protocol to monitor is disabled

  1. User enabled a specific job:

<174>Nov 05 09:55:59

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2

,[phCustId]=1,[customer]=Super,[jobName]=System cpu usage,

[srcIpAddr]=192.168.20.164,[appTransportProto]=SNMP,[sessionId]=111

78d2aeae08e9c2babe2725fa1,[procName]=AppServer,[hostIpAddr]=192.168

.64.124,[hostName]=ACCELOPS-W2K3B4, [eventSeverity]=PHL_INFO,[jobStatusType]=UserEnabledJob,[user]=admi n,[pullInteval]=180,[phLogDetail]=Protocol to monitor is enabled

  1. User changed job polling interval:

<174>Nov 05 09:57:21

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[phEventCategory]=2

,[phCustId]=1,[customer]=Super,[jobName]=System real memory usage,

[srcIpAddr]=192.168.20.164,[appTransportProto]=SNMP,[sessionId]=111

78d2aeae08e9c2babe2725fa1,[procName]=AppServer,[hostIpAddr]=192.168 .64.124,[hostName]=ACCELOPS-W2K3B4,[eventSeverity]=PHL_INFO,[jobSta tusType]=UserChangedPollIntv,[user]=admin,[pullInteval]=300,[phLogD etail]=Interval of protocol to monitor is changed

<174>Nov 05 10:33:01

[PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[errReason]=Missing

/Invalid WMI credential for 192.168.20.207, PROC_RESOURCE, [phEventCategory]=2,[phCustId]=1,[jobId]=1545818,[customer]=Super,[ jobName]=Process Resource Usage via WMI, [srcIpAddr]=192.168.64.153,[appTransportProto]=WMI,[sessionId]=13b3 48ad44270e0249eafc9dfdc5,[procName]=AppServer,[hostIpAddr]=192.168.

20.207,[hostName]=win-li5sipp8s7s.accelops.net,[eventSeverity]=PHL_

INFO,[jobStatusType]=DiscoveryNotScheduled,[user]=1,[pullInteval]=1

80,[phLogDetail]=Monitor on device is not scheduled

  1. Successful job:

<174>Nov 05 10:13:00 [PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[errReason]=,[phEve ntCategory]=2,[phCustId]=1,[jobId]=1536112,[customer]=Super,[jobNam e]=Process Resource usage via SNMP,[srcIpAddr]=192.168.64.153,[appTransportProto]=SNMP,[sessionId

]=128e159a35b9f0f4cd71ca80222b,[procName]=AppServer,[hostIpAddr]=19

2.168.20.170,[hostName]=qa-win2008-217.accelops.net,[eventSeverity]

=PHL_INFO,[jobStatusType]=ExecutionSuccess,[user]=1,[pullInteval]=1

20,[phLogDetail]=Status of monitor is changed by Job

  1. Failed job:

<174>Nov 05 10:15:00 [PH_AUDIT_DEV_MON_JOB_STATUS_CHANGE]:[custId]=1,[errReason]=Failed to get process utilization in

executeGeneralProcResourceJobOpt,[phEventCategory]=2,[phCustId]=1,[ jobId]=1536112,[customer]=Super,[jobName]=Process Resource usage via SNMP, [srcIpAddr]=192.168.64.153,[appTransportProto]=SNMP,[sessionId]=12a b68da5ca2cceab2a69cbda16e,[procName]=AppServer,[hostIpAddr]=192.168 .20.170,[hostName]=qa-win2008-217.accelops.net,[eventSeverity]=PHL_ INFO,[jobStatusType]=ExecutionFailed,[user]=1,[pullInteval]=120,[ph

LogDetail]=Monitoring device failed

  1. Job stays in “Discovered Added” state for more than 15 minutes and is not scheduled:

<174>Nov 05 10:55:57 [PH_AUDIT_DEV_MON_JOB_NOT_STARTED]:[custId]=1,[phEventCategory]=2,[ phCustId]=1,[customer]=Super,[jobName]=ICMP Ping Status,[appTransportProto]=PING,[procName]=AppServer,[hostIpAddr]=1

72.16.10.110,[hostName]=HOST-172.16.10.110,[eventSeverity]=PHL_INFO

,[jobStatusType]=DiscoveryAdded,[user]=SYSTEM(phDiscovery),[pullInt eval]=120,[phLogDetail]=Monitoring job did not start yet

Enhanced custom parser development graphical user interface

The custom parser development graphical user interface is enhanced to include the following:

  1. Ability to search text within the XML file.
  2. Add a line number in the XML file. When there is a Error in ‘Validate’ or ‘Test’, show the line number as a reference to help user fix the problem.
  3. Allow user to reformat the text after block update for easy readability.
  4. Allow an option to Clear XML in one shot to allow for bulk replace.
  5. Color code the XML tags and text for easy readability.
  6. Show the parsed fields in Test results in a nice tabular form.
  7. Improve the scrolling/editing response for large XML files.
  8. Show the XML in a tree form – allow cross-linking of the XML Tree and the text edit window.
  9. Allow user to increase the size of the edit window.

Fixed Issues and Minor Enhancements

General GUI

Bug 7489: Created a CMDB report named “Active Dependent Rules” that tracks which rules depend on other rules. This helps users to tweak/enable/disable chained system rules

Bug 8021: Added indices in ph_task and ph_alter tables in PostgreSQL – this improves the GUI experience when user visits Alert and

Task tabs

Bug 8054: Allow an option to search on ‘Origin’ field in every tab in Admin > Device Support area. This allows users to quickly see user defined Device/Application types, Event Types, Event Attribute Types, Parsers and Dashboard columns

Bug 8165: Show VLAN as a column in Analytics > Identity and Location Report

Bug 8181: Need to get result of scheduled report even if the report has no data

Bug 8291: Allow user to unlock an AccelOps account

Bug 8896: Allow scheduled reports to skip charts and only contain tables

Bug 9266: Added “errReason” attribute to system event PH_REPORT_ACTION_STATUS – the attribute states why notification failed

Bug 9670: CMDB Device shows under Scheduled Maintenance even after device removed from Schedule Maintenance Calendar

Bug 9900: Expose Last Updated Time and Discover Method fields of a device for use in CMDB reports

Bug 10083: Display a warning when user disables or deletes a rule that is referenced in other rules

Bug 10172: Change the AccelOps GUI CMDB > Users so that all locally created users cannot edit the “domain” field

Bug 10198: Removal of devices or organizations from CMDB sometimes display foreign key violation errors

Bug 10250: Remove “Show Password” check box for credential

Bug 10382: enhancement:  allow HTML tags in custom e-mail templates

Bug 10394: Allow bulk disabling for blocked IP in the CMDB through the GUI

Bug 10450: Add ‘Apply To’ option to facilitate applying multiple authentication profiles to one or more users

Bug 10563: Add an Export button for Related Incidents screen

Bug 10830: Add locations view in summary dashboard

Bug 11343: Long device names truncated on Widget dashboard

Bug 11371: Allow import/ export of user defined watch list

Bug 11498: A rule with CLEAR conditions becomes invalid after clone process – constraints between main and clear rules are not properly copied over

Bug 11508: Ability to set locations for a large number of devices in  one shot

Bug 11596: Ability to add Notes to Rule exceptions. One should also be report on Rule exceptions.

Bug 11597: Add Remediation section to Rule definition. Add this to default email template. Make this part of CMDB report. Add this to custom notification template.

Bug 12583: User can not manually add important processes that have the same name but different process parameters

Bug 12613: Columns on Amazon EC2 performance view should be same as EC2 dashboard

Bug 12694: Provide an option to not have charts in exported PDF reports

Bug 12698: Enable search on “Monitor Errors” and “Error Description” on Admin > Setup wizard > Monitor Change/Performance > Monitor Errors popup

Bug 12760: Edit a Report Schedule and the Report automatically Runs

Bug 12786: Make error message clearer for event dropping rule creation on grouped incidents

Platform

Bug 9518: Glassfish log rotation is now configured for saving space – only keep 20 files and each of them max-sized 2000000 bytes.

Bug 9828: EPS Pulling functionality has limitations that lead to dropping of events by collectors

Bug 9938: Allow modular ‘yum upgrades’ for non-base-CentOS packages like JVM, Chrome, PostgreSQL, Glassfish

Bug 10144: Do not overwrite customer’s ssl.conf during upgrade

Bug 11926: DNS caching code has performance issues

Bug 12130: AccelOps uses rsyslog to receive our internally generated events. There is a throttle defined there (200 messages in 5 minutes interval). This will result message loss in high throughput situation like VoIP phone discovery, Layer 2 port mapping discovery etc. This throttle is removed since this is intra-computer communication and can handle much higher message rates.

Bug 12538: Detailed events, rules and reports for performance monitoring status changes

Bug 12584: Collectors sometimes fail to negotiate HTTP(S) connection to Super/Worker if ever they choose SSLV3 (because of poodle vulnerability – possibly because of a man-in-the-middle device like a IPS or a firewall disallowing all SSLV3 negotiations.

Bug 12585:The phMonitor module crashes when it sees a 3.7.6’s rest_cache_api list entry in phoenix_config.txt

Bug 12586: The configuration file phoenix_config.txt needs to be upgraded properly by maintaining user’s changes from previous versions

Bug 12644: Run script notification may fail if the raw message contains special XML characters

Bug 12649: Updating Dynamic Watch List by incidents causes Application Server to run out of memory when there are many many-to-many relationships between incidents and dynamic watch lists

Bug 12650: Full VM build does not ‘yum update’ packages as previously designed

Performance Monitoring / Event Pulling / Synthetic Transaction Monitoring (STM)

Bug 9848: Packet transmission timeouts for SNMP v1 and v2 phoenix_config needs to be extended from 1 minute to 5 minutes

Bug 11423: Add Custom command output monitoring via winexe for windows environments

Bug 12066: Parse CVSS_BASE score for vulnerabilities into (vulnCvssBaseScore attribute)

Bug 12213: Cisco IOS CPU can not be monitored in some cases with multiple CPUs – performance monitoring has to identify the control plane cpus

Bug 12214: PerfMonitor module will stop sending PH_DEV_MON_PING_STAT events for a gateway if its immediate down steam device are down

Bug 12387: NexPose vulnerability report XML parsing takes a long time

Bug 12458: Checkpoint needs a resume event handler

Bug 12561: Discovery never removes a PING job even if the device is not reachable by PING during discovery. This is done since a

PING is fundamental for measuring up time. Only a user can manually disable PING jobs

Bug 12601: Admin > Setup wizard > Monitor Change/Performance tab does not reflect the status of successful discovery after correcting device credentials.

Bug 12604: Events are not picked up by parser module if Supervisor node is down for an extended period of time

Bug 12625: For Qualys Vulnerability Scanner, Test connectivity succeeds, but the discover method and event pull methods are not set in discovery, resulting in no job creation for Java agent.

Bug 12661: Don’t trigger config change while getting config error

Bug 12730: Allow pulling interval to be less than 1 minute to pull windows logs at a faster rate. Added a phoenix config entry of

“wmi_pull_interval_scale” that can have a range from 1 (default) to 10. If users want to have a shorter interval for WMI event pulling, they can change it to 6 to make it 6 times faster; if the pulling interval in GUI is 1 minute, then events are pulled every 10 seconds.

Bug 12754: Enhance custom command output monitoring to generate an event to indicate no matching lines for regular expression. This can be used to detect e.g. a process is down from running the top command. If the regular custom command output monitoring command is PH_DEV_MON_CUST_DF then AccelOps would generate PH_DEV_MON_CUST_DF_NOT_FOUND when the are no matching lines in the command output

Bug 12787: Enhance performance monitoring status job upload to keep uploading if failed last time

Bug 12801: Custom command output monitoring – deleted items are still monitored

Bug 12803: Custom SNMP job monitoring sometimes fails to distinguish when one key is a prefix of the other; e.g.

SNMPv2-SMI::enterprises.9.9.48.1.1.1.5.1 from SNMPv2-SMI::enterprises.9.9.48.1.1.1.5.16. So the obtained value for

SNMPv2-SMI::enterprises.9.9.48.1.1.1.5.1 may be the value for SNMPv2-SMI::enterprises.9.9.48.1.1.1.5.16

Bug 12804: Custom Performance Monitoring – when custom transforms are nested, the order may not be preserved resulting in wrong calculations. For example, if custom transforms are defined as “used = transform(used/1024)”, “total = transform(total/1024)” and “memUtil = transform(used/total)”, then the transforms must be calculated in the order used -> total -> memUtil. Earlier releases did not do this.

Rule / Query / Report Engine

Bug 8512: Turn off inaccurate system rule ‘Windows Disk controller problem’

Bug 9847: Add Identity and location to CMDB Report

Bug 10996: Worker down rule did not trigger after license expires

Bug 11008: RFE:  when retrieving Triggered Events for an Incident in GUI or e-mail notification, search 60 minutes before and after

Incident Time. This is fixed by the ‘Trigger Event Query Optimization’ feature

Bug 12322: Increase per rule GROUP BY thresholds – per cust and over all customers

Bug 12558: Improved cache miss handling for profile anomaly rule handling: The profiles for anomaly detection are stored in a SQLite database. When rule engine attempts to look up the average and standard deviation values for a particular key (e.g. IP address, port number etc), and an exact match is not found, then earlier releases picked up the lowest values for that profile in that database. This heuristic often causes unnecessary rule triggers. This release makes tightens the cache miss handling case – profile anomaly rules do not trigger for a key value if the average and standard deviation values are not found for that key value for that time period.

Bug 12640: When performing analytical searches when individual countries from the country groups are referenced in filter conditions as objects, then no results are returned

Discovery

Bug 10363: Disallow discovery via Virtual IP

Bug 10533: Add ability to define IP subnets in the 172.16.16.0/22 type format

Bug 11308: Foundry router becomes generic-generic

Bug 11972: Telnet discovery of HP Procurve switches fails due to error in expect script

Bug 12713: If host name contains unprintable characters like backspace(x08) and enquiry(x05), then parsing this XML causes app server to throw exceptions and run out of memory

Device Support

Bug 9942: RFE: add performance monitoring for Cisco WAPX (lightweight) devices

Bug 10006: RFE:  add device support for Tripp lite UPS devices

Bug 10307: Microsoft cluster services incorrectly assigned to Microsoft Exchange Application Group

Bug 10362: Add support for (Oracle) Acme Packet Border Controller

Bug 10366: Support for Dell Force 10 Networking devices

Parsing  area

Bug 8894: Cisco ASA parser: trailing white space in User attribute causing searches to miss events with condition User EQUALS ‘string’ Bug 10418: Parse username in Windows MSSQL Event 18453

Bug 11351: Parse username in Win-Security-5145

Bug 12485: Parse jitter field in Cisco VoIP CDR/CMR record

Bug 12626: Snort events collected via database have wrong severity

Bug 12641: Added more Symantec Anti-virus events

Bug 12643: A null pointer exception can happen during pulling performance monitor config for discovery

Bug 12656: Some FortiGate traffic logs do not parse if “status=” is missing in the logs

Bug 12668: More Foundry switch logs to be parsed

Bug 12714: Enhance McAfee EPO parser to parse more logs

Bug 12716: Enhance Cisco IOS and NX-OS parser to parse more logs

Bug 12757: Put Fortigate firewall DHCP messages into the identity and location section – so the IP to user and host name mapping shows up when FortiGate is acting as the DHCP server

Bug 12765, 12767: Add a “Total Bit Rate” attribute for interface utilization and Netflow events – so user can quickly run 95th percentile on the total bandwidth for an interface

Data: System Rules/Reports

Bug 8512: Turn off inaccurate system rule ‘Windows Disk controller problem’

Bug 10113: Added description for windows security events: 5142, 5143, 5144, 5168, 4985, 5145

Bug 12660: Enhance “Heavy TCP Port Scan” rule to exclude Windows Security Firewall logs (Win-Security-5156), ASA/PIX Teardown events (ASA-302014, PIX-302014, FWSM-302014)

Bug 12676: The event type group for JUNOS_KMD_VPN_DOWN_ALARM_USER event is incorrect

 

FortiSIEM What’s new in Release 4.2.3

$
0
0

What’s new in Release 4.2.3

This release addresses several issues and enhancements on top of 4.2.2 release.

Note: To upgrade to this release, migrate to 4.2.1 first and then upgrade to 4.2.3. It is not possible to directly upgrade from 3.7.x or earlier releases to 4.2.2 because of the Linux Operating System changes.

The following are the key issues that are addressed in this release:

Bug 9211: Windows log and metric pulling via WMI has been optimized to be roughly 4x faster and more robust

Bug 11459: Checkpoint LEA log collection agent occasionally restarts

Bug 11631: VMware log pulling sometimes stops after encountering an exception in the API

Bug 11699: QueryMaster module memory grows to be large when there are over large number of devices (over 2500) in summary dashboard.

Other addressed issues include the following:

General GUI related fixes and enhancements

Bug 11353: App server stops picking discovery result xml files when malware site/IP auto update failed

Bug 11517: Windows server version in device selector UI can not be seen completely

Bug 11526: Delete custom event attribute or custom event type – the custom dashboard column not removed

Bug 11586: Footer shows wrong language when browser’s OS language is not in English

Bug 11654: Custom Property Attribute not populating in query conditions

Bug 11655: CMDB/Performance page shows CPU green at 100% utilization

Bug 11735: App Server Exception for incidents with custom event attribute causes performance issues

Platform related fixes and enhancements

Bug 11435: Handle error: “File does not exist: /var/www/html/favicon.ico”

Bug 11574: Include misc debugging tools: nfsiostat, iostat, screen, ntop

Bug 11812: Custom group is not editable after migration

Performance Monitoring / STM related fixes and enhancements

Bug 11336: Add sent bits/sec and received bits/sec to Netflow metrics

Bug 11410: Nessus vulnerability scanner reports are not parsed correctly

Bug 11422: Add “Diff” system transform for custom SNMP and WMI performance monitoring

Bug 11426: Possible memory leak issue in VMware performance pulling agent

Bug 11428: Add use cases for Linux syslog monitoring – detect “yum update”, system going down, network interface up/down, process killed by kernel because of out of memory

Bug 11449: The NMAP check during Flow based service detection can cause unnecessary probing traffic from AccelOps. Remove the nmap test or make it optional.

Bug 11450: VMware monitoring enhancements:

Add Cluster names and VCenter IP to all VMware host events

Add Folder to show VM performance metrics on the right hand side in VMware view

Add folder information to VMware guest and ESX events

Capture VMware tools version including it is out of date or not Add cluster balance information

Bug 11470: Exclude Mounted Volumes from disk space monitoring at client machine. The mounted volume disk space has to be monitored at the server side.

Bug 11620: Some interfaces (like Serial) have send/recv bytes only from regular IF-MIB and not from high speed MIB; so need to pick interface speed from regular IF-MIB and not from high speed MIB. Currently speed for T1 interfaces is picked from high speed MIB and so it is 2Mbps instead of 1536Mbps.

Rule / Query / Report Engine related fixes and enhancements

Bug 10934: The rule “Concurrent VPN Authentications To Same Account From Different Cities” need to be enhanced to cover the case where user attribute is not present in the log.

Bug 11360: Some pre-defined rules does not map Reporting IP to Destination IP in incident events – this may cause notification policy to trigger

Bug 11456: Include Reporting IP for consideration in Notification policy > Affected Objects. Currently Affected Object check includes only Destination IP and Host IP.

Bug 11483: Rule synch – new worker causes performance  issues

Bug 11594: Should restart phRuleMaster when failed to retrieve rule exception

Bug 11775: Incident fails to trigger when host name contains special characters which are not acceptable in XML e.g. &, <, > etc

Parsing related fixes and enhancements

Bug 10418: User name in Windows MSSQL Server Event 18453 is not parsed

Bug 11230: Certificate Information in Win-Security-4768 and Win-Security-4771 not parsed

Bug 11239: Event time order not always maintained at the Supervisor/Worker nodes

Bug 11280: FortiGate event “FortiGate-traffic-icmp-allowed” is improperly classified as a denied event and triggers rules

Bug 11466: Several events are not parsed for Barracuda Mail gateway

Bug 11473: If parser sets event severity, then let it win over event severity from syslog header

Bug 11615: Juniper SSL VPN parsing extensions

Bug 11634: User not parsed in Windows Security 4625 events

Bug 11696: For Cisco ASA, Network Interface > Security Level info not consistently propagated to parser – this causes problems in identifying source and destination interfaces for parsing network traffic

Discovery related fixes and enhancements

Bug 11260: CBQoS / BGP / OSPF metrics get falsely discovered for Cisco devices even when they are not configured on the device

Bug 11397: Allow HTTPS selection choice for NetApp ONTAPI discovery

Bug 11519: Update access IP after re-discovery if original access IP interface is down

Bug 11524: Handle “>” as prompt in Unix SSH scripts

Bug 11582: EMC VNX CPU Discovery fails with password special characters

Bug 11681: EMC VNX discovery fails when it has only Meta LUNs but no normal LUNs

Bug 11755: VMware VCenter 5.5 discovery cannot return the correct tree structure when a Data Center is created under a Folder

Device Support

Bug 11419: VMware VCNS log parsing

Bug 11474: Collect Back-to-back consistency point metric for NetApp from SNMP

Bug 11539: Support for Emerging Threats Snort rules

Bug 11553: Match Cisco MARS SIM rules

Bug 11559: NeXpose Rapid7 XML Export 2.0 Report format not supported

Bug 11570: Support FireEye HX appliance

Bug 11616: IronPort-Web Parser Logic Error

Bug 11680: Parse additional foundry syslog

Bug 11697: Add user name and source IP addr from ASA-113019 into identity and location report

Bug 11722: User information in Spanish win-security-4625 cannot be parsed correctly

Bug 11727: Update Cisco IPS Signatures to latest

Bug 11732: Windows events 673, 4769, 4773, 674, 677, 4770 are assigned to wrong log failure group causing brute force logon rules to fire

Bug 11733: Windows Clustering Failover rule definition needs to be tightened by adding  the constraint eventSource = “Microsoft-Windows-FailoverClustering”

FortiSIEM What’s new in Release 4.2.2

$
0
0

What’s new in Release 4.2.2

This release fixes several issues and adds several enhancements on top of 4.2.1 release.

 

Note: To upgrade to this release, migrate to 4.2.1 first and then upgrade to 4.2.2. It is not possible to directly upgrade from 3.7.x to 4.2.2

because of the operating system changes.

 

General GUI related fixes and enhancements

Platform related fixes and enhancements

Performance Monitoring / STM related fixes and enhancements

Rule / Query / Report Engine related fixes and enhancements

Parsing related fixes and enhancements

Discovery related fixes and enhancements Open Issues

 

General GUI related fixes and enhancements

Bug 5532: User can write duplicate event forwarding rules

Bug 8100: Under CMDB > Blocked IP addresses > Emerging Threats, the last updated time stays at 1969 even after setting up the “Update Automatically”

Bug 9023: For Selenium based Web STM, the Selenium script upload file feature should report an error message when user doesn’t select a file

Bug 10279: Exporting and then importing back the same report creates two reports

Bug 10306: Remedy incident clear time is incorrect in AccelOps

Bug 10741: Adding a Selenium script definition using Edit/Paste fails with run time error: Could not find Firefox in your system Path.

Bug 10816: Clone Event Attribute Type has not Value Type

Bug 10848: Null column header shows in report when exporting incident “Incident Notification Error”

Bug 10880: In Analytics > Generated Reports, a user with  read only view privilege should not be able to delete a report

Bug 10972: Maintenance calendar month view should display as March 2014 instead of 03,2014

Bug 10993:  Load Report page is not paginated – loads slowly

Bug 11017: User with edit and run privilege can not export Identity and Location Report

Bug 11027: User with View and Run privilege should not be able to Import Rules

Bug 11046: GUI allows multiple organizations without collectors with overlapping IP address ranges

Bug 11047: Incident notification via email: Incident details incorrectly shows Triggered Event Count  instead of Incident Count

Bug 11051: Clicking Related Incidents for “Excessive Denied Connections From An External Country” shows errors

Bug 11067: Test Connectivity button does not work but the drop down menu works

Bug 11072: Schedule field in CMDB report on Report does not support multiple records

Bug 11178: Imported custom dashboard column can not show in org view

Bug 11264: Add “free disk” to Exec Summary and All Device dashboard

Bug 11306: Allow other Flow sources like SFLOW, ASA Netflow in CMDB > Interface Stats > Inbound/outbound flow drill down   a

Selenium scrip

Platform related fixes and enhancements

Bug 11122: Incident notification via SNMP and HTTP(S) fails on VA mode

Bug 11127: Notification action is successful but the Incident Notification Status column is empty

Bug 11168: Incidents which belong org with collector can display in orgs without collector on incident dashboard calendar view page Bug 11184: System error “succeed ratio too low” isn’t cleared automatically Bug 11286: Upgrade to CentOS 6.5

Performance Monitoring / STM related fixes and enhancements

Bug 11053: Capture reserved disk size for Linux disk space monitoring

Bug 11195: Incorrect User Connections information on MySQL dashboard

Bug 11221: Linux disk space monitoring (via SSH) does not work for Debian Linux

Bug 11305: Remove PH_DEV_MON_CUST restriction from Custom performance jobs – this allows new device type’s CPU, Memory to be shown in dashboards

Bug 11332: Faulty Hardware monitoring – if failed once – then never reattempted again

Rule / Query / Report Engine related fixes and enhancements

Bug 11246: Unable generate reports using Network Segment folders

Parsing related fixes and enhancements

Bug 11099: Parse PostFix SMTP gateway logs

Bug 11149: Need to alert on Microsoft Cluster Service Failure errors

Bug 11153: Add parsing for Symantec IDS events

Bug 11167: Incorrect error handling for XML parsing by the parser module

Bug 11177: Need to set event severity from syslog priority field

Bug 11201: Fortinet parser extensions to cover more event parsing

Bug 11222: Clone and Test CiscoIPSParser does not work

Discovery related fixes and enhancements

Bug 11193: CMDB reports wrong memory unit for EMC VNX and Clarion

Bug 11232: Merge across Collectors incorrect in some cases – we need enhancement to merge same host across collectors so long they belong to same organization

Bug 11233: CMDB view is incorrect for VCenter discovered VMs when multiple guests on common ESX is split across customers

Bug 11236: Nx-OS interface speed incorrect when an interface has both ifHighSpeed and ifSpeed entries

Bug 11244: Need to add Windows 2008 R2, Windows 2003 R2, Windows 2012 R2 as new device types

Bug 11261: Cisco IOS router discovery crashes in certain cases with Cisco VoIP entries

Bug 11263: Show datastores for ESX during VCenter/ESX discovery

Bug 11264: Detailed Linux device type discovery using SSH – replace General Linux with Redhat Linux, Ubuntu Linux etc

Bug 11277: Remove extra “System Reserved” disk for Windows via WMI

Open Issues

 

FortiSIEM What’s new in Release 4.2.1

$
0
0

What’s new in Release 4.2.1

This release adds features and functionality in several areas.

Systems Features

Fundamental system upgrade

Run on Microsoft HyperV

Regex based search

Statistical anomaly detection

Ability to drop events

Enhanced PDF export

Remote connectivity to collector devices

CMDB device discovery filter

Delta discovery

Query/Rule usability enhancements

Support OpenLDAP for user discovery and external authentication

Support secure LDAP protocols: LDAPS and LDAP Start TLS CMDB Report extensions

Performance and Availability Monitoring

Customizable high performance summary dashboards

Generalized Selenium based Web Synthetic Transaction Monitoring (STM)

End-to-end Email Synthetic Transaction Monitoring (STM)

Custom SSH based command output monitoring

Log Management and Security Incident Event Monitoring (SIEM)

Watch list

Agent less file change and file integrity monitoring

Log integrity validation

Compliance reporting

Device Support

Miscellaneous Key Enhancements

Quick interface utilization drill down via Netflow

Manually define trunk ports

Add CMR Support for Cisco VoIP

Limit excessive generated incidents

Cisco ASA/IOS Remote Access VPN Monitoring

Most Expensive Query report

VM Snapshot monitoring Excluded Disks

General GUI related fixes and enhancements

Platform related fixes and enhancements

Performance Monitoring / STM related fixes and enhancements

Rule / Query / Report Engine related fixes and enhancements

Parsing related fixes and enhancements

Discovery related fixes and enhancements Open Issues

 

Systems Features

Fundamental system upgrade

This release upgrades the AccelOps platform to CentOS 6.4, Glassfish 3.2, Postgres 9.1 and JDK 1.7. This significantly enhances the stability and robustness of the base operating system and key sub-systems. In addition, the Apache web server is also upgraded to 2.2.25 to fix many vulnerabilities.

Run on Microsoft HyperV

In addition to VMware ESX, Redhat KVM and Amazon EC2, AccelOps supervisor, worker and collectors will now also run on Microsoft HyperV virtualization platform.

Installing AccelOps Supervisor, worker and collector on Microsoft HyperV is covered here.

Regex based search

Prior to Release 4.2.1, AccelOps allows the ability to search raw messages by AND/OR combination of keywords and via the CONTAINS and NOT CONTAINS operator. This release adds the ability to search via regular expressions (REGEXP operator). Regular expressions can return precise search results than keyword based searches. Regular expressions can be used in searches, queries and rules for any string valued attribute such as Event Type, Raw Message etc.

Regex based search for real time search is covered here.

Regex based search for historical search is covered here.

Regex based search for rules is covered here.

Statistical anomaly detection

This release enables users to baseline any performance metric or flow data collected by AccelOps and create an alert when the current value of the metrics deviates significantly from its statistical baseline. Baselines are created on an hourly basis for each distinct hour of the day and separately for work and non-work days – a total of 48 buckets. This fine-grained approach allows for accurate behavioral analysis. The baselines are learned automatically and updated continuously in an attempt to learn the new normal. The following baselines are available out of the box

Network Traffic Analysis

Network traffic – by sender, by receiver, by connection – total flows, sent bytes, received bytes, sent packets, received packets

Firewall TCP/UDP port traffic – by inbound,  by outbound – permitted traffic and denied traffic – total flows Firewall total denied flows

VPN usage – by user – time spent and total traffic volume

Host resource usage – CPU, Memory, Virtual Memory, Disk I/O

Application resource usage – CPU, Memory

Network interface usage – by interface – utilization, sent bytes, received bytes, sent errors, received errors SAN usage – LUN I/O

NAS usage – Volume usage, Protocol latency

DNS requests – by sender – requests, unique resolution requests

ICMP requests – by sender – requests, unique destinations

Web requests – count, errors, distinct clients

Reporting eps

Login – successful count, failed count

Server process count

Login – successful count, failed count

Reported Event types

Reported Errors

User seen – distinct count

Statistical anomaly detection is covered here.

Ability to drop events

Certain devices and applications generate a significant of logs. Often logs are very verbose and certain log types are of little value and waste valuable storage. This release provides the ability for user to drop the events immediately after they are accepted by AccelOps. These logs do not count towards licensed eps and do not trigger rules.

Ability to drop events is covered here

Enhanced PDF export

The charting technology in PDF reports is significantly enhanced – the exported charts now look similar to what the users see in GUI.

Remote connectivity to collector devices

After detecting a problem, often there is a need to open a session to a monitored device either via Telnet/SSH, VNC, HTTP(S), Microsoft RDP directly from AccelOps GUI. This release release enables AccelOps users to launch terminal sessions to monitored devices for the following two cases

device is in the same data center as Supervisor/Worker

device is in a remote date center behind a firewall and monitored by a Collector. This case uses a reverse SSH tunnels between Collector and Super

The topic of opening remote connections to collector devices is covered here.

CMDB device discovery filter

This release enables users to control the CMDB device discovery process by including or excluding certain device types. For virtualization discovery, powered off VMs and VM templates can also be excluded. This facilitates a “clean” CMDB consisting of only the devices of importance to the user.

Setting up CMDB device discovery filters is discussed here.

The following reports in CMDB > Reports > System Audit can be used to report on devices added or deleted to CMDB via discovery

CMDB: Device Addition and Deletion History

CMDB: Device Modification History

Delta discovery

Because of the depth at which AccelOps discovers a device, a full discovery of a range of devices can take some time. Often there is a need to quickly detect only the new devices in the network. This is an important PCI compliance requirement – for security purposes, it is critical to know if there are any new devices plugged in to the network. This release enables users to quickly discover devices and applications that are not already in AccelOps CMDB.

Setting up delta discovery is discussed here.

The following reports in CMDB > Reports > System Audit can be used to report on devices added or deleted to CMDB via discovery

CMDB: Device Addition and Deletion History

CMDB: Device Modification History

Query/Rule usability enhancements

This release improves the Query/Rule usability by creating the following shortcut operations.

Ability to turn a query to a rule (see here) and vice versa (see here)

Ability to convert a historical search to a real time search and vice versa (see here) Ability to smart copy a query from one tab to another (see here)

Ability to create display and filter templates and use them later in a query or rule (see here)

Support OpenLDAP for user discovery and external authentication

This release extends user discovery and external authentication capabilities from Microsoft Active Directory to also include OpenLDAP servers.

Setting up OpenLDAP based discoveries and external user authentication is covered here.

Support secure LDAP protocols: LDAPS and LDAP Start TLS

This release extends secure LDAP protocols: LDAPS and LDAPStartTLS for user discovery and external authentication. This topic is covered here.

CMDB Report extensions

Currently users can create CMDB reports for exporting CMDB information. Currently this includes a report of devices in CMDB, their installed and running applications, hardware inventory etc. This release extends this capability to include users, rules and reports. Specifically users can now run reports on

Discovered users

AccelOps administrative users and their roles

Active Rules with exceptions if any

Scheduled Reports

Active Performance Monitors

For details see here

The following inbuilt reports in CMDB Report section can be used to quickly get relevant information.

Discovered Users

Externally Authenticated AccelOps Users

Locally Authenticated AccelOps Users

Manually Defined Users

Active Rules

Inactive Rules

Rules with Exceptions

Scheduled Reports

Active Performance Monitors

Performance and Availability Monitoring

Customizable high performance summary dashboards

Summary dashboards, a unique AccelOps feature, provide a real-time bird’s cross-domain metrics and health of a device or a group of devices or applications. There are 3 types of dashboards:

single level e.g. All Network dashboard, Hardware summary

two-level dashboards e.g. database performance dashboard, ESX-VM dashboard

three level dashboards e.g. Business Service dashboard (Business Service -> Devices -> Applications), VM Cluster dashboard (Cluster -> ESXs -> VMs)

In this release, all of these dashboards are enhanced for scalability and extensibility:

the dashboards are now configurable users can create their own dashboards

users can add their own performance metrics whether they are collected by default or created by customers as custom monitors paginated dashboard views eliminate the limitation of 300 devices/applications per dashboard – searching and sorting happens across the entire set of devices, not just on the page the user is on

most performance metric computations (like maximum or average interface utilization over all interfaces of a device) are now shifted from the GUI to the Super/Worker cloud – this dramatically reduces GUI network bandwidth and improves GUI rendering speed.

The ability to customize a summary dashboard is covered here.

Generalized Selenium based Web Synthetic Transaction Monitoring (STM)

The ability to monitor websites by running complex multi-level synthetic monitoring tests is an important criterion. This release allows AccelOps customers to record any web transaction from a web browser via Selenium plugin, and then play it back within AccelOps framework for continuous monitoring. Alerts can be triggered when the script fails to run or has an unacceptably large delay.

For setting up a Selenium based Web STM, see here.

End-to-end Email Synthetic Transaction Monitoring (STM)

For properly testing the health of email system, it is important to be able to test the entire path of an email: sender -> SMTP gateway -> receiving SMTP gateway -> receiving mailbox. This release provides AccelOps users a framework to run end-to-end e-mail synthetic tests, e.g. AccelOps will send an email and make sure that the same email is received within an acceptable time limit. Alerts can be triggered when an email fails to arrive or has an unacceptably large delay.

For setting up an end-to-end email STM, see here.

Custom SSH based command output monitoring

Often customers have scripts that monitor certain aspects of a system or an application and there is a need to get those script outputs into AccelOps for reporting and alerting. This release provides customers a framework for running these scripts remotely, bringing back the command output via SSH, parsing the output and creating events for further analysis in AccelOps.

For setting up command output monitoring, see here.

Log Management and Security Incident Event Monitoring (SIEM)

Watch list

This release enables users to create and manage watch lists. Watch lists are (often dynamic) containers that can hold objects of interest, e.g. Network Scanners, Frequent Locked out users, Externally excessive denied ports, High I/O Virtual machines etc. Watch lists can be dynamically populated when a rule triggers. An entry can leave the watch list if it does not trigger in a defined period of time or the entry could be permanent. A watch list can also be populated statically. A watch list can be further used in a rule condition in a nested manner to trigger rules of significant more significance. Watch lists provide a easy way to keep track of important policy violators in a monitoring system without always running reports.

For setting up command output monitoring, see here.

Agent less file change and file integrity monitoring

Unauthorized and untested configuration changes often lead to critical failure conditions. AccelOps already provides a way to keep track of network device configuration changes. This capability is further extended in this release to address the following situations – in an agent less fashion.

File integrity via checksum for specific files or directories on a server – trigger an alert when there is a change in any file in any directory.

The files need to be accessible via SSH or Telnet.

Monitor the content of a file on devices and make sure that it is identical to a target “blessed” file. Alert when the monitored file differs from the target file and show exactly what has changed.

Setting up agent-less file change/integrity monitoring is discussed here.

Log integrity validation

This release enables AccelOps customers to demonstrate to security auditors that the collected logs have not been tampered with, while at rest within AccelOps monitoring system. To achieve this, logs are cryptographically signed immediately upon arrival at the AccelOps point of entry. Using the AccelOps GUI, the cryptographic checksum can be validated to prove to the auditors that logs have not been tampered with. In case the logs have indeed been tampered with, AccelOps can identify the time period of the affected logs.

Log integrity validation is discussed here.

Compliance reporting

AccelOps already provides compliance reports for the following standards: PCI, HIPAA, COBIT, GLBA, FISMA, NERC, GPG13. This release extends this set by providing compliance reports for the following standards

SANS Critical Controls ISO

Device Support

Checkpoint Provider-1 – collect  firewall logs from CLM and audit logs from MDM – see here for details

MySQL database – discovery, performance monitoring and audit log collection and analysis – see here for details

IBM DB2 Audit Log  – audit log parsing and analysis –  see Configuring Security Gateways for details

Cisco AVC – log analysis via Netflow

McAfee Foundstone Vulnerability Scanner – full log parsing and analysis  – see Web Server Configuration for details HyperV log parsing via Honeycomb agent

Windows log parsing and file integrity monitoring via Honeycomb agent

Windows log parsing via Correlog agent

Dell Blade center

EMC Data Domain

Citrix NetScaler performance monitoring

Link interface errors (1.3.6.1.2.1.10.7.2)

EMC Isilon

Alcatel AAA Radius syslog

Miscellaneous Key Enhancements

Quick interface utilization drill down via Netflow

This release enables users to quickly analyze a network interface usage issue by combining SNMP and Netflow. SNMP provides interface utilization metrics and Netflow provides the traffic on that interface. AccelOps makes this connection seamless assuming that the router/switch is monitored by SNMP and also sending Netflow to AccelOps.

To achieve this drill down

Go to CMDB, select the router and click ‘Interface Stats’. The displayed data is from SNMP.

Note that for any interface, there is a drill down for Inbound or Outbound traffic. Select the desired direction and the interface usage for the chosen direction is displayed. This information is gathered from Netflow.

Manually define trunk ports

User identity and location is a key feature in AccelOps. To accomplish this, AccelOps automatically discovers switch trunk ports to ensure that only the access ports and not trunk ports show up in Identity and Location reports. An exception is of course VoIP ports which have PCs connected to them. This release enables to manually label certain discovered interfaces as trunk ports. Future discoveries will take this input into consideration and not create Identity and location entries for those user defined trunk ports.

To label a discovered interface as a trunk port, go to CMDB; choose a device; ‘Edit’ interfaces; check the Trunk Port checkbox and click Save. You need to do a discovery again to get new identity and location information.

Add CMR Support for Cisco VoIP

Cisco VoIP Call information is in two files

CDR records – this contains primarily the Call originator and Call destination information CMR records – this contains call quality information – MOS scores

Prior to this release, AccelOps only handled CDR records. This release is able to “join” CDR and CMR records to append the call quality information to the call originator/destination information in a single event.

Limit excessive generated incidents

 

Cisco ASA/IOS Remote Access VPN Monitoring

 

Most Expensive Query report

As part of database performance monitoring, AccelOps can now monitor the most expensive queries. Currently it works for Oracle and MS SQL Server.

 

Two rules are provided that trigger when a query takes more than 5 minutes to complete. The query has to complete for the rule to trigger.

VM Snapshot monitoring

VM Snapshots consume lots of space. This release allows you monitor the space taken up by snapshots

Excluded Disks

Often there are certain disk volumes that are either read only or always close to full and never grow. Because of these disks, these servers always as CRITICAL in dashboards. This release enables users to exclude these disks from monitoring – details are here.

Fixed Issues

General GUI related fixes and enhancements

Bug 7580: While trying to validate a custom parser (cloned), user gets “Backend error code: 139” and cannot continue working on parser Bug 7589: Remove 300 device limitation in custom dashboard

Bug 7648: Devices do not update after moving Organizations

Bug 7713: Chart is not displayed in report if generate the report with “Display as” columns.

Bug 7780:  CSV exported report does not need extra commas

Bug 7958: Dashboard displays error when Request XML is over 20MB

Bug 8119: Clear Reason for system cleared has incorrect “active for more than 7 days”

Bug 8166: When creating a Ticket the “Assigned To” drop down contains ALL LDAP users, not just AO users

Bug 8170: Add search box on floating dialog boxes showing Performance Monitor errors

Bug 8180: Customer ID of All Report Notification report shows Super always

Bug 8683: Dashboard for Oracle does not display instance status

Bug 8713: Need to allow to import downloaded malware domain csv file manually

Bug 8719: Display IPS Generator ID and IPS Signature ID in wrong format

Bug 8725: “Last Update” of malware domain/blocked IP is not updated

Bug 8736: Ability to set values for ports, applications, device types at global level in CMDB that apply to all orgs

Bug 8762: If you schedule a report and set a custom value in Maximum graphs or Maximum rows and save it then go back into scheduled reports, it goes back to default values. It appears graphs/rows are saved with the custom values because scheduled reports work correctly.

Bug 8768: After changing the Font – some words do not fit correctly

Bug 8826: Customer added a second email address in Admin -> General Settings -> Analytics. Scheduled Report emails are not being sent to the second address

Bug 8946: Org Users should not  be able to see Systems Errors from other Orgs Bug 8962: Allow for bulk delete of report results.

Bug 8963: Perf Incidents are not displayed properly from Dashboard (Exec. Summary)

Bug 8989:  Disk utilization drill-down dialog stop working once browser loose focus

Bug 9072: When editing credentials of various protocols, they all display the ‘description’ field. SNMP v3 protocol does not display this field. This led users to incorrectly use the context field to to put description which caused failures.

 

Bug 9073: When looking at a widget – in this case “Top Incidents by Severity, Count”, and you drill down into an incident, you are brought to the incident page with a list of the incidents – BUT – after sorting the widget, and drill down on the same incident, you are show an incorrect incident

Bug 9096: Parse out additional fields in Symantec AV events

Bug 9116: Remove SMS notification configuration from user guide

Bug 9182: Clear incident notification email contains incorrect host name which belongs to another org with the same IP

Bug 9311: Too many tasks causes GUI to be slow

Bug 9332: The report output limitation is incorrect in the pop up report run dialog

Bug 9374: Maintenance Calendar does not save Devices folders

Bug 9391: Notification policy does not work when set only the days of the week and the start date, toggle the days of the week

Bug 9539: Need to forbid “Test Rule” button for rules from organizations

Bug 9628: Daylight saving time (DST) causes report editing to not properly re-save time

Bug 9675: For Juniper SRX devices that have virtual interfaces with a 192.0.0.0 subnet mask, user cannot edit and modify any CMDB fields for this device

Bug 9734: Read-Only Admin Role allows credential to IP association edits with double click

Bug 9839: Provide ability to export IP Range to Credential associations. Currently, the Export button on Admin->Setup->Credentials only exports the credentials but not the IP to Credential associations

Bug 9892: When export CMDB report run for 1 Org, you get data for ALL Orgs

Bug 9953: Deleting Clear Conditions in Rule do not work

Bug 10023: In Dashboard > widgets > combo view,  y -axis has no scale value

Bug 10046: Last updated method overwrites the health page. Discover first using WMI then VM_SDK and you get just the VM stats Bug 10161: When SVN has too many configuration/installed software revisions on single device, UI got timed out by 120 seconds.

Finally, UI shows error and cannot see device configuration

Bug 10186: Email notification contains another organizations Interface description

Bug 10202: AO-SP: Rule Sync Errors dialog show other organization’s error

Bug 10215: AO-SP: For Rule Synch errors, an Org user should be able see ONLY his own changes causing synch errors. Super/global should be able to see all Organization errors

Bug 10263: LDAP discovery does not add users when there is an exception caused by the address field is too large to fit into our 256 character column field on ph_contact

Bug 10288: For Report Bundles, report end times are off by 1 minute

Bug 10292: For Report Bundles, absolute schedule time not saved when custom email notification is chosen

Bug 10344: Prevent user from entering more characters then allowed in text fields in UI. When creating a device maintenance schedule, entering too many characters in the description field resulted in “transaction marked for rollback” message

Bug 10673: Custom group does not show up in role management UI Access tree. This issue is affecting to many places such as Dashboard, CMDB, query condition builder

Bug 10829: When selecting multiple devices in CMDB for setting up maintenance schedule the devices do not have name in dialog box. This is GUI issue only.

Platform related fixes and enhancements

Bug 7600: Provide a CMDB Report for Active Rules for an Organization

Bug 7680: External authentication does not work with OpenLDAP

Bug 7953: Events dropped based on Elastic EPS being too slow in changing values

Bug 8142: Event Packager port unnecessarily open on the super and workers

Bug 8213: Back end modules should check certificates during SSL communication

Bug 8278: Script for creating bluecoat ftp directory does not change owner.group to ftpuser.ftpuser

Bug 8279: Every time AO device is rebooted bluecoat ftp directory owner.group are reset to admin.admin preventing files from being ftp’ed to AO

Bug 8650: Need to consolidate NSCD package to nscd-2.5-24 with its updated configuration Bug 8676: Proxy configuration does not work

Bug 8718: Add CMDB Report for Users in AccelOps

Bug 8869: Provide capability in CMDB reports to extract intersection (AND) of 2 criteria – when the criteria partially overlap

Bug 8983: Need to auto-compact JMS request queue

Bug 9065: Empty username and password in the base URL definition causes upgrade failed

Bug 9184: Support CS MARS type drop rules

Bug 9283: Documentation for Amazon AWS need to say Access key and Secret access key instead of User ID and Password

Bug 9341: Global EPS license is ignored when Elastic EPS allocates EPS based on previous configuration

Bug 9630: Create a report for all users with system roles in AccelOps

Bug 9669: Limit the amount of times that AppServer retries a transaction

Bug 9928: Create a report for Monitor Change Performance errors

Bug 9937: Do not overwrite phoenix_config.txt upon upgrade

Bug 10009: Create a report to show all processes and open ports

Bug 10143: Customer certificate overwritten in ssl.conf with 3.7.6 upgrade

Bug 10223: Cache files created by incident notification are never removed

Bug 10525: Provide an option to not show the Domain drop-down list from logon page for external authentication Bug 10567: Fix customer found Apache security vulnerabilities

Performance Monitoring / STM related fixes and enhancements

Bug 7787: Add Virtual Memory Utilization Attributes support

Bug 8254: False high Exch Metrics

Bug 8700: Monitor VM snapshots

Bug 8837: AO is losing checkpoint firewall events every time phCheckpoint crashes

Bug 8919: The average processing time for Glassfish servlets and processors are wrong

Bug 8981: Windows log pulling time interval not implemented correctly

Bug 9137: Add load balancing metrics for Cisco ACE load balancer

Bug 9458: Need instance availability monitoring via Amazon AWS SDK

Bug 9508: Add performance monitoring for Citrix NetScaler

Bug 9627: “PH_DB_DATA_ERROR: cannot decrypt password for principal” exception thrown when running perfMonitor rest API on

Super in SP mode when there are devices monitored by a collector

Bug 10028: Performance monitor job status does not update (shows yellow star)

Bug 10222: Enhance custom performance monitor to include string index (currently only integer index)

Bug 10249: Cisco ASA CPU, memory utilization not populated correctly

Bug 10471: NX-OS devices PH_DEV_MON_INTF_UTIL not calculated correctly sometimes

Bug 10512: Apache metrics are not being pulled

Bug 10527: Fail to collect UDP echo IP SLA stats

Bug 10608: Monitoring Windows network interface statistics via WMI sometimes crashes when server has more than 1 network interface

Bug 10618: Hardware Monitoring for Cisco IOS and NX-OS devices stops after 3.7.6 upgrade

Bug 10650: Important process with long parameters (like java) are not being detected as down when they are indeed down

Bug 10827: Enhance Linux disk utilization by accounting for reserved space

Bug 10921: Generate per-host LUN usage metrics for EMC VNX/Clarion

Rule / Query / Report Engine related fixes and enhancements

Bug 7509: Prohibit user from choosing past dates in scheduled reports

Bug 8376: Reports saved as csv have blank “Event Name” column

Bug 8811: Query returns nothing for user defined port/protocol group

Bug 9405: Analytics can only query top level CMDB Application Groups

Bug 10183: When running a long report, App server needs to retry if Query Master does not respond

Bug 10345: Report Scheduler not sending reports at the correct time

Bug 10348: Daylight Savings Time (DST) causes report editing to not properly re-save time

Bug 10349: Quartz worker thread pool are too small to handle large set of scheduled reports

Bug 10487: Enhance CONTAIN operator to match anywhere in string – not just in the beginning Bug 10767: Eliminate Rule synch errors caused by Rule exception

Parsing related fixes and enhancements

Bug 7625: Suppress certain events from being parsed

Bug 7639: Windows Parser does not completely parse when key pair fields are in Spanish

Bug 8216: Parse out additional fields in ASA-722051 event

Bug 8355: Allow event forwarding based on event severity

Bug 8702: Event Type Win-Security- 4656 does not parse object type and object name

Bug 8758: Symantec AnitVirus logs – virus file name not being parsed

Bug 8874: Fix Microsoft UAG parser

Bug 8889: Fix Forescout CounterACT parser

Bug 8895: Allow parsing of “:” in certain Cisoc IOS messages

Bug 8999: Windows WMI OS Parser does not properly parse all fields on SQL Server Events

Bug 9024: Windows Parser needs to strip the @Domain.xxx at the end of “account name” in the key-value-pair

Bug 9026: from Admin->Device Support->Event Type, if you click on the amplifier icon on the search box and uncheck ‘Device Type’, it ignores the search columns settings which always includes Device Type as part of search. Actually it ignores whatever is unchecked, it always displays the default

Bug 9451: WatchGuard Firewall Parser Missing Events

Bug 9587: Email subject is sometimes not parsed by Cisco IronPort Mail parser

Bug 9641: Save event name from Cisco IPS Alerts

Bug 9709: Add support for Alcatel AAA Radius syslog

Bug 9843: NetApp snnp traps not parsed

Bug 9861: Change Parser to not create User= “-” for event type IIS-Web-Client_Access_Denied

Bug 9862: F5 BIG-IP LTM new version has syslog not parsed by our current parser

Bug 9955: Cisco switch events for dynamic ARP inspection (SW_DAI-4-DHCP_SNOOPING_DENY) are not being fully parsed

Bug 10090: Cisco ASA Parser enhancement –  add the type of Remote Access connection that is used when a DAP is applied

Bug 10091: Parse WLAN AP Host Name attribute for Cisco WLAN disassociation SNMP traps

Bug 10092: Enhance Cisco Call Manager CDR parser to get the CM Login User ID of the caller

Bug 10119: Logon Fail events misclassified as Login Success events

Bug 10286: AccelOps is not parsing Severity and Source / Destination addresses, Event Type, Event Name, etc. from Snort Sensor IDS logs, because of the <br0> between the [Priority 1]: and {TCP}

Bug 10324: Parse addition fields for Symantec Endpoint Protection (SEP) logs

Bug 10381: Fix JunOS log parsing errors

Bug 10408: Parse logs generated by Correlog windows agent

Bug 10418: Parse user name from Windows MSSQL Event 18453 Bug 10458: Convert TOS values to DSCP values in Netflow

Bug 10511:  Parse out Account Name and Object Name for Win 4670 events

Discovery related fixes and enhancements

Bug 6574: Provide an option for removing VM monitoring

Bug 6955: Support secure LDAP protocol

Bug 7888: Checkpoint running SPLAT OS being incorrectly discovered as Generic Linux

Bug 8681: Redhat 10GB Interface discovered as 10MB

Bug 8769: Support MySQL – discovery, performance monitoring and log collection

Bug 8984: Compress Discovery result before sending to App server

Bug 9042: MS SQL Server JDBC discovery fails but AO still pulls some perf metrics but not all This is unique to MS SQL Server 2012.

Bug 9058: LDAP discovery using “daysToPasswordExpiry” is completely wrong

Bug 9070: Add SNMP support for Dell Blade Center / Chassis Mgmt Controller

Bug 9134: Support EMC Data Domain

Bug 9290: Add region to AWS EC2 discovery

Bug 9420: Checkpoint discovery does not have Installed Software/Running Process on GAIA/Security Platform

Bug 9474: VM discovery does not work with symbol character for password

Bug 9584: Make the ‘show ip route’ command optional in discovery of Border routers with millions of routes

Bug 9790: VMSDK discovery failure does not show complete failure reason

Bug 9827: Option to include or exclude VM Guest hosts during ESXi / VMSDK discovery

Bug 10025: Add device support for standalone Cisco WLAN AP (not controllers) running IOS

Bug 10171: Cisco IPS module in ASA has wrongly discovered IPS SW version

Bug 10437: Cannot discover Cisco device by SSH with high privilege user

Bug 10524: Discovering powered off VMs may cause incorrect merge if the VMs have a shared IP address

Bug 10530: Linux interface speed incorrect

Bug 10677: Interface alias not discovered when an interface has more than one addresses

 

Open Issues

Bug 7537: Can not create new incident category in VA mode. AO-SP works correctly. Workaround is to manually add the incident category to a rule in the database.

Bug 7191:  Device Maintenance window cannot be larger than 1 day

 

 

 

FortiSIEM Basics

$
0
0

FortiSIEM Basics

These topics provide an overview of the FortiSIEM solution, including its component and various deployment configurations.

Supervisors, Workers, Collectors, and Organizations

Enterprise Deployment Options

Standalone Supervisor Deployment for Enterprises

Supervisor and Worker Cluster Deployment for Enterprises

Supervisor with Collectors Deployment for Enterprises

Matrix of Enterprise Deployment Configuration Options

Multi-Tenant Deployment Options for Managed Service Providers or Multiple Organizations

Standalone Supervisor Deployment for Multi-Tenancy

Supervisor and Worker Cluster Deployment for Multi-Tenancy

Supervisor with Collectors Deployment for Multi-Tenancy

Matrix of Multi-Tenancy Deployment Configuration Options  Export Restrictions

FortiSIEM Features and Architecture

$
0
0

Features and Architecture

FortiSIEM provides an all-in-one, seamlessly integrated and service-oriented IT infrastructure monitoring solution that covers performance, availability, change, and security monitoring aspects of network devices, servers, and applications. It is offered in two versions:

A VMware based virtual appliance, which you can deploy as a single appliance or a cluster of virtual appliances in a highly available, scaled-out grid architecture. This is what we refer to as FortiSIEM Enterprise.

Software-as-a-Service (SaaS), where you deploy a Collector virtual on-premises for a customer, and all of the customer data is transmitted to an FortiSIEM data center. This is what we refer to as FortiSIEM Multi-Tenant, since collector deployments are commonly used by organizations such as Managed Service Providers to monitor the services of their customers.

Some of the features of the FortiSIEM monitoring solution include:

Intelligent Device Discovery

Analytics

Business Services

Architecture

Intelligent Device Discovery

The first step in the monitoring process is IT infrastructure discovery. FortiSIEM has a fast and intelligent discovery engine that can automatically crawl an IT infrastructure and discover network devices, servers, and applications in depth. The user needs to provide appropriate credentials, a discovery IP address range, and optionally a starting router IP address for faster discovery.

A wide range of information is discovered including hardware information, serial numbers and licenses, installed software, running applications and services, and router configuration. The discovered devices are automatically categorized into detailed functional groups, such as Routers/Switches, Firewalls, and Network IPS, and this information is maintained within an integrated configuration management database (CMDB). Some special relationships are also discovered, for example WLAN Access Points to WLAN Controllers, VMware guests to physical hosts, etc. The CMDB is kept up to date through user-defined scheduled discoveries and FortiSIEM listening to changes as part of performance monitoring.

A novel aspect of FortiSIEM discovery is that those aspects of a device that can be monitored are also discovered at the same time. For example, given SNMP, WMI, and JDBC credentials for a Windows server, FortiSIEM might discover the following:

System performance metrics that can be collected by SNMP, for example CPU, memory utilization, and disk space utilization

System performance metrics that can be collected by WMI, for example Disk I/O utilization, memory swap rates, and process utilization Application specific metrics that can be collected by WMI, for example IIS, DNS, DHCP, and Exchange metrics Event logs that can be collected by WMI

Database logs that can be pulled from the server by JDBC

You simply approve the discovered results and monitoring begins. This approach reduces human error, since FortiSIEM learns from the true device configuration state.

Analytics

FortiSIEM uses a unified event-based framework to analyze all data including logs, performance monitoring data. Logs can either be sent to FortiSIEM via Syslog, SNMP traps, or other common log shipping methods, or FortiSIEM can periodically access the system and collect the logs. Performance monitoring data is collected by periodically probing the system. The data is parsed, indexed, and stored in a proprietary flat-file based database. In contrast, the CMDB information is stored in a PostgreSQL relational database. FortiSIEM unified data management architecture combines the two databases and presents a single view to the user.

FortiSIEM provides a broad range of metrics. First, it is possible to search all data based on keywords or in a structured way using the attributes parsed by AcceOps. The search can be done in real time, in which the data streaming in from devices is displayed, or the search can be based on historical data. Historical data is referred to as Reports in FortiSIEM, and can be scheduled to run at intervals you set. A large number of reports are provided in a categorized fashion, based on device type, and also based on functionality such as availability, performance, change and security. Two novel aspects of FortiSIEM metrics include unification and drill-down capabilities. With unification, all the data is analyzed and presented the same way, whether it be real time search, reports, rules or performance, availability, or change or security data. By using drill-down you can start from a specific context, such as Top Authentication Failed Users, and iteratively select attributes to further analyze data and get to the root cause of a problem. As an example, the investigation of Top Authentication Failed users could follow a drill-down of pick user and time range -> Top Destination IP, Ports for specific user and time range -> pick destination IP and port -> Query all raw messages.

FortiSIEM also uses rules for real-time alerting – a real-time event correlation engine analyzes all data and triggers alerts based on these rules. FortiSIEM ships with 500+ broad rules that cover a broad range of inter-related performance, availability, change and security scenarios. Rules can vary from simple text search and threshold conditions, to comprehensive logic supporting full Boolean operators and nested sub-patterns referencing multiple elements including thresholds and defined services. Thresholds can be static or dynamically derived from profiled network, system resource and user activity. You can add new rules, and customize existing ones, as described in Creating Rules using GUI. Business Services

A business service lets you view FortiSIEM metrics and prioritize alerts from a business service perspective. A business service is defined within FortiSIEM as a smart container of relevant devices and applications serving a business purpose. Once defined, all monitoring and analysis can be presented from a business service perspective. It is possible to track service level metrics, efficiently respond to incidents on a prioritized basis, record business impact, and provide business intelligence on IT best practices, compliance reporting, and IT service improvement. What is also novel about FortiSIEM is how easily a business service can be defined and maintained. Because FortiSIEM automatically discovers the applications running on the servers as well as the network connectivity and the traffic flow, you can simply choose the applications and respective servers and be intelligently guided to choose the rest of components of the business service. This business service discovery and definition capability in FortiSIEM completely automates a process that would normally take many people and considerable effort to complete and maintain.

Architecture

The FortiSIEM virtual appliance solution operates as a turnkey, guest host application running within the most popular hypervisors with the option of using NFS or local storage. The implementation process is flexible and can be accomplished in phases to support a variety of distributed and hybrid-cloud implementations The FortiSIEM virtual appliance is placed on a network where it can obtain operational data, as well as establish sessions with the infrastructure. Remote sites can use the FortiSIEM Collector client to locally discover, collect, compress and securely transmit of operation data back to the FortiSIEM virtual appliance. FortiSIEM’ scale-out architecture allows for virtual appliance clustering to increase processing capacity and availability. Additional virtual appliances can be added on-the-fly with nominal configuration, which will automatically distribute workload across cluster members to extend event analysis throughput and to reduce query response time.

 

 

Page 91


FortiSIEM Supervisors, Workers, Collectors, and Organizations

$
0
0

Supervisors, Workers, Collectors, and Organizations

An FortiSIEM deployment can be configured using either a single virtual appliance, or with multiple virtual appliances that play different roles within the deployment. The Supervisor virtual appliance is the primary component in both standalone and cluster deployments, and all deployments begin with the set up and configuration of the Supervisor. As described in Supervisor and Worker Cluster Deployment for

Enterprises, there may be situations in which the single appliance cannot monitor all the data and devices in your infrastructure, and so you can deploy Worker virtual appliances to take up the extra load. Finally, you may encounter situations in which you need to deploy Collectors  for the purpose of gathering data that will be processed by Supervisors and Workers. As described in Supervisor with Collectors Deployment for Enterprises and Supervisor and Worker Cluster Deployment for Multi-Tenancy, these are most likely situations where you need to monitor IT infrastructure for different sites, as in the case of a large or distributed enterprise, or for different organizations, as in the case of multi-tenant installations for Managed Service providers (MSPs). For these situations each Organization is defined separately within FortiSIEM, so you can tailor your monitoring, analytics, and reports to meet the specific needs of that organization.

 

FortiSIEM Deployment Options

$
0
0

Deployment Options

FortiSIEM architecture of workers, collectors, and supervisors offers a number deployment options for enterprises at any level of scale, as well as deployment options for managed service providers who need multi-tenant solutions. Topics in this section describe these deployment options in detail, including use cases for each deployment type as well as node and server configurations for each deployment type.

Enterprise Deployment Options

Standalone Supervisor Deployment for Enterprises

Supervisor and Worker Cluster Deployment for Enterprises

Supervisor with Collectors Deployment for Enterprises

Matrix of Enterprise Deployment Configuration Options

Multi-Tenant Deployment Options for Managed Service Providers or Multiple Organizations

Standalone Supervisor Deployment for Multi-Tenancy

Supervisor and Worker Cluster Deployment for Multi-Tenancy

Supervisor with Collectors Deployment for Multi-Tenancy

Matrix of Multi-Tenancy Deployment Configuration Options

Enterprise Deployment Options

For FortiSIEM, an Enterprise deployment is one in which there is a single organization for which data is gathered and analyzed, and the virtual appliances are located entirely on-premises for that organization.

Standalone Supervisor Deployment for Enterprises

Supervisor and Worker Cluster Deployment for Enterprises

Supervisor with Collectors Deployment for Enterprises

Matrix of Enterprise Deployment Configuration Options

Standalone Supervisor Deployment for Enterprises

This is the simplest possible deployment option, in which a single Supervisor handles all the work of monitoring, processing, and analyzing data.

You can configure the Supervisor to use local or NFS storage, depending on your event data storage requirements, as described in Using NFS

Storage with AccelOps

Supervisor and Worker Cluster Deployment for Enterprises

As the number of monitored devices, or the analyzed event rate, grows, one Supervisor may not be able to handle the load. In that case, you can deploy a cluster of Supervisor and Worker virtual appliances that share data over NFS. In a cluster deployment, the Supervisor and Worker nodes have specific functions:

Supervisor with Collectors Deployment for Enterprises

$
0
0
Supervisor with Collectors Deployment for Enterprises

There are two cases where a single Supervisor may not be enough for your deployment.

There are monitored devices behind a firewall that will not allow monitoring protocols like Windows Management Instrumention (WMI) to be used from the Supervisor

The Supervisor can only reach the monitored devices through a high latency network like a Wide Area Network (WAN), in which case monitoring like protocols like Simple Network Management Protocol (SNMP) or WMI do not work well

In these cases you can deploy Collectors to monitor the devices, and they will communicate to the Supervisor over HTTP(S). The Collectors communicate with the devices, collect and parse events and logs, compress them, and then send them to the Supervisor for monitoring and analysis. Collectors also can buffer the events, in case transmission to the Supervisor is interrupted. As shown in the diagrams, you can use Collectors in a deployment with a single Supervisor, or in a deployment that also includes Workers.

An AccelOps deployment with a single Supervisor and Collectors

An AccelOps deployment using a Single Supervisor + 2 Workers + 2 Collectors

Matrix of Enterprise Deployment Configuration Options

This matrix shows the components required for each enterprise deployment option.

Deployment Option Supervisor

Node

Worker

Node

Collector

Node

NFS

Server

Report

Server

Visual

Analytics

Server

Description
Single Supervisor Node         x           This is the most basic single site enterprise deployment.
Supervisor Node with

Collectors

        x          x       This is also an enterprise deployment covering multiple sites. Data collection is simplified by deploying a collector for the satellite sites.
Enterprise Cluster         x         x        x     This is the scalable enterprise deployment. An NFS Server is required in the data sharing architecture between Supervisor and Worker nodes.
Enterprise Cluster with

Collectors

        x         x        x      x     This deployment adds collectors to the mix and is the most comprehensive enterprise deployment.
Supervisor Node with

Tableau Visual Analytics

        x          x  x This is the most basic single node enterprise deployment, with added capability for Visual Analytics with Tableau
Supervisor Node with

Collectors and Tableau

Visual Analytics

        x          x      x  x This is also an enterprise deployment covering multiple sites with added capability for Visual Analytics with Tableau. Data collection is simplified by deploying a collector for the satellite sites.
Enterprise Cluster with Ta bleau Visual Analytics         x         x        x    x  x This is the scalable enterprise deployment with added capability for with added capability for Visual Analytics with Tableau. An NFS Server is required in the data sharing architecture between Supervisor and Worker nodes.
Enterprise Cluster with

Collectors and Tableau

Visual Analytics

        x         x        x      x    x  x This deployment adds collectors to the mix and is the most

comprehensive enterprise deployment, with added capability for Visual Analytics with Tableau.

 

Multi-Tenant Deployment Options for Managed Service Providers or Multiple Organizations

$
0
0

Multi-Tenant Deployment Options for Managed Service Providers or Multiple Organizations

While a common use case for FortiSIEM is the monitoring of IT infrastructure for a single enterprise, Managed Service Providers (MSPs) and large enterprises with multiple organizations can also use FortiSIEM to monitor IT infrastructure at the customer or organization level, either by splitting IP addresses to correspond to the customer or organization, or by deploying Collectors for each customer or organization and managing the monitoring and analysis of their data from a centralized Supervisor.

Standalone Supervisor Deployment for Multi-Tenancy

Supervisor and Worker Cluster Deployment for Multi-Tenancy

Supervisor with Collectors Deployment for Multi-Tenancy

Matrix of Multi-Tenancy Deployment Configuration Options

Standalone Supervisor Deployment for Multi-Tenancy

FortiSIEM allows users to create organizations, to and manage the entire IT infrastructure monitoring life cycle from data collection, storage, analytics and alerting for an organization that organization as a separate entity from other organizations. There are several use cases for this this multi-tenant model.

Hosting service providers that host multiple customers in their own data center

Managed service providers that manage a customer’s data centers from their own data center

Large enterprises that want to manage separate parts of the organization as individual customers

The simplest multi-tenancy deployment involves a single Supervisor, with organizations defined through the splitting of IP address ranges. For example:

Page

Supervisor and Worker Cluster Deployment for Multi-Tenancy

As the number of monitored devices, or the analyzed event rate, grows, one Supervisor may not be able to handle the load. In that case, you can deploy a cluster of Supervisor and Worker virtual appliances that share data over NFS. In a cluster deployment, the Supervisor and Worker nodes have specific functions:

10.1.2.0/24 = Customer 2

During the discovery process, the AccelOps Supervisor node will tag a device with the correct customer ID based on the IP address definition.

 

Matrix of Multi-Tenancy Deployment Configuration Options

$
0
0
Matrix of Multi-Tenancy Deployment Configuration Options

This matrix shows the components required for the each multi-tenancy deployment option.

Deployment Option Supervisor

Node

Worker

Node

Collector

Node

NFS

Server

Report

Server

Visual

Analytics

Server

Description
Single Multi-Tenant

Supervisor Node

        x           This is the most basic single site multi-tenant deployment, primarily suitable for hosting providers. Organizations are created by splitting up the IP address space.
Multi-Tenant Supervisor

Node Collectors with

        x          x       This is a service provider deployment covering multiple sites. Data collection is simplified by deploying a collector for the satellite sites. You can add organizations by assigning a collector to an organization, or by splitting up the IP address space.
Multi-Tenant Cluster         x         x        x     This is a scalable service provider deployment suitable for deployments with large compute and storage needs. An NFS Server is required in the data sharing architecture between Supervisor and Worker nodes. Organi zations are created by splitting up the IP address space.
Multi-Tenant Cluster with

Collectors

        x         x        x      x     This deployment adds collectors to the configuration and is the most comprehensive service provider deployment. You can add organizations by assigning a collector to an organization, or by splitting up the IP address space.
Multi-Tenant Supervisor

Node with Tableau Visual

Analytics

        x          x  x This is the most basic single site multi-tenant deployment, with added capability for Visual Analytics with Tableau.
Multi-Tenant Supervisor Node with Collectors and T ableau Visual Analytics         x          x      x  x This is a service provider deployment covering multiple sites, with added capability for Visual Analytics with Tableau. Data collection is simplified by deploying a collector for the satellite sites.
Multi-Tenant Cluster with T ableau Visual Analytics         x         x        x    x  x This is a scalable service provider deployment ,with added capability for Visual Analytics with Tableau. An NFS Server is required in the data sharing architecture between Supervisor and Worker nodes.
Multi-Tenant Cluster with

Collectors and Tableau

Visual Analytics

        x         x        x      x    x  x This deployment adds collectors to the configuration and is the most comprehensive service provider deployment, with added capability for Visual Analytics with Tableau.

 

 

FortiSIEM Installation

$
0
0

Installation

Additional Information in the Help Center

You can find additional information about installation, upgrades, and license management for your AccelOps deployment in the Installati on, Upgrades, and Licenses section of the Help Center maintained by AccelOps Support.

The topics in this section are intended to guide you through the basic process of setting up and configuring your AccelOps deployment. This includes downloading and installing the AccelOps OVA image, using your hypervisor virtual machine manager to configure the hardware settings for your AccelOps node, setting up basic configurations on your Supervisor node, and registering your Supervisor and other nodes. Setting up IT infrastructure monitoring, including device discovery, monitoring configuration, setting up business services, is covered in under the section Confi guring Your AccelOps Platform.

What You Need to Know before You Begin Installation What Kind of Deployment Will You Set Up?

Who Will Install and Configure AccelOps?

What Information Do You Need to Get Started? The Basic Installation Process

What You Need to Know before You Begin Installation

What Kind of Deployment Will You Set Up?

Before beginning installation you should have determined the exact deployment configuration you will follow, as described in the topics under Dep loyment Options. Note that many deployment options have particular hardware requirements. For example, if you intend to use an NFS server for a cluster deployment, or if want to use Visual Analytics, you will need to make sure that you have the necessary hardware and network components in place. We strongly recommend that you read through all the installation topics for your deployment configuration before you begin.

Who Will Install and Configure AccelOps?

These topics assume that you have the basic system administration skills required to install AccelOps, and that you are already familiar with the use of hypervisors such as VMware ESX or, if you are setting up a Cloud deployment, that you are already familiar with Cloud environments such as Amazon Web Services.

What Information Do You Need to Get Started?

You will need to have administrator-level permissions on the host where you will download and install AccelOps, and you will also need to have username and password associated with your AccelOps license. If you intend to use NFS storage for event data, you will also need to have set up an NFS server prior to installation.

The Basic Installation Process

The installation process for any AccelOps deployment consists of a few steps:

Import the AccelOps virtual appliance into a hypervisor or Amazon Web Services environment

Edit the virtual appliance hardware settings

Start and configure the virtual appliance from the hypervisor console

Register the virtual appliance

Topics in this section will take you through the specific installation and configuration instructions for the most popular hypervisors and deployment configurations.

System Performance Estimates and Recommendations for Large Scale Deployments

Browser Support and Hardware Requirements

Information Prerequisites for All FortiSIEM Installations

Hypervisor Installations

Installing in Amazon Web Services (AWS)

Determining the Storage Type for EventDB in AWS

Configuring Local Storage in AWS for EventDB

Setting Up Supervisor, Worker and Collector Nodes in AWS

Setting Up AWS Instances

Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS Configuring the Supervisor and Worker Nodes in AWS

Registering the Collector to the Supervisor in AWS

Setting up a Network Bridge for Installing AccelOps in KVM

Importing the Supervisor, Collector, or Worker Image into KVM Configuring Supervisor Hardware Settings in KVM

Importing a Supervisor, Collector, or Worker Image into Microsoft Hyper-V

Setting the Network Time Protocol (NTP) for ESX

Installing a Supervisor, Worker, or Collector Node in ESX

Importing the Supervisor, Collector, or Worker Image into the ESX Server

Editing the Supervisor, Collector, or Worker Hardware Settings

Setting Local Storage for the Supervisor

Troubleshooting Tips for Supervisor Installations

Configuring the Supervisor, Worker, or Collector from the VM Console

ISO Installation

Installing a Collector on Bare Metal Hardware

General Installation

Configuring Worker Settings

Registering the Supervisor

Registering the Worker

Registering the Collector to the Supervisor

Using NFS Storage with AccelOps

Configuring NFS Storage for VMware ESX Server

Using NFS Storage with Amazon Web Services

Setting Up NFS Storage in AWS

Setting Up Snapshots of EBS Volumes that Host EventDB and CMDB in AWS

Moving CMDB to a separate Database Host

FortiSIEM Windows Agent and Agent Manager Install

FortiSIEM Windows Agent Pre-installation Notes

Installing FortiSIEM Windows Agent Manager

Installing FortiSIEM Windows Agent

 

System Performance Estimates and Recommendations for Large Scale Deployments

$
0
0

System Performance Estimates and Recommendations for Large Scale Deployments

This topic includes estimates and recommendations for storage capacity, disk performance, and network throughput for optimum performance of FortiSIEM deployments processing over 10,000 EPS.

In general, event ingestion at high EPS requires lower storage IOPS than for queries simply because queries need to scan higher volumes of data that has accumulated over time. For example, at 20,000 EPS, you have 86,400 times more data in a day than in one second, so a query such as ‘Top Event types by count for the past 1 day’ will need to scan 20,000 x 86,400 = ~ 1.72 billion events. Therefore, it is important to size your FortiSIEM cluster to handle your query and report requirements first, which will also handle event ingestion very well. These are the top 3 things to do for acceptable FortiSIEM query performance:

  1. Add more worker nodes, higher than what is required for event ingestion alone
  2. 10Gbps network on NFS server is a must, and if feasible on Supervisor and Worker nodes as well
  3. SSD Caching on NFS server – The size of the SSD should be as close to the size required to cache hot data. In typical customer scenarios, the last 1 month data can be considered hot data because monthly reports are quite commonly run

 

Schedule frequently run reports into the dashboard

If you have frequently run ranking reports that have group-by criteria (as opposed to raw message based reports), you can add such reports into a custom dashboard so that FortiSIEM schedules to run these reports in inline mode. Such reports compute their results in streaming manner as event data is processed in real-time. Such reports do not put any burden on the storage IOPS because they read very little data from the EventDB. Note that raw message reports (no group-by) are always computed directly from EventDB

 

An example scenario is presented at the end of this guide.

System

Performance

Component

Estimates and Recommendations
Event Storage

Capacity

Storage capacity estimates are based on an average event size of 64 compressed bytes x EPS (events per section). Browser Support and Hardware Requirements includes a table with storage capacity requirements for up to 10,000 EPS.
Root Disk

IOPS

 Standard hard disk IOPS
CMDB Disk

IOPS

1000 IOPS or more. Lab testing for EC2 scalability used 2000 IOPS.
SVN Disk

IOPS

1000 IOPS
EventDB

IOPS for

Event

Ingestion

1000 IOPS for 100K EPS (minimum)
EventDB Read IOPS for Queries As high as feasible to improve query performance (use SSD caching on NFS server when feasible). In EC2 scalability testing, 2000 read IOPS while ingesting 100K EPS using one supervisor and two workers produced these results:

Index Query – No filter, display COUNT(Matched Events), group-by event type for 24 hours

1.  Total Events processed = 2,594,816,711 (2.59 billion events)

2.  Average events per second scanned by Query (QEPS) = 1.02 million QEPS

3.  Average Query Runtime = 2543 seconds (~ 42 minutes)

Raw Event Log Query – Same as Index Query with filter Raw Event Log contains ‘e’

1.           Total Events processed = 350,914,385 (350 million events)

2.           Average events per second scanned by Query (QEPS) = 179,909 EPS (179k QEPS) 3.  Average Query Runtime = 1950 seconds (~ 33 minutes)

Network

Throughput

Recommend 10Gbps network between Supervisor, Workers, and NFS server.

Using VMXNet3 Adapter for VMware

To achieve the best network throughput in VMware environments, delete the E1000 network adapter and add one that uses VMXNet3 for theeth0/eth1 network configuration. VMXNet3 adapter supports 10Gbps networking between VMs on the same host as well as across hosts, though you must also have a 10Gbps physical network adapter to achieve that level of throughput across hosts. You may need to upgrade the virtual hardware version (VM Ware KB 1003746) in order to have the ability to use VMXNet3. More details on different types of VMWare network adapters is available in VMWare KB 1001805

Achieving 10Gbps on AWS EC2

To achieve 10Gbps in the AWS EC2 environment, you will need to:

1.  Deploy FortiSIEM Super, Workers, and NFS server on 8xlarge class of instances (for example, c3.8xlarge ). Refer to EC2 Instance Types for available types, and look for instance types with 10 Gigabit noted next to them.

2.  You will need to use the HVM image for both the FortiSIEM image and NFS server image that supports enhanc ed networking.

3.  Supervisor, Workers, and NFS Server must be placed under the same AWS EC2 placement group within an AWS VPC.

Network

Interfaces

FortiSIEM recommends the use of separate network interfaces for event ingestion/GUI access and storage data to NFS
Number of

Workers

6000 EPS per worker for event ingestion. More worker nodes for query performance. See example below.

 

Example:

An MSP customer has 12,000 EPS across all their customers. Each event takes up 64 bytes on average in compressed form in the EventDB.

These calculations are just extrapolations based on a test on EC2. Actual results may vary from this because of differences in hardware, event data, types of queries. Therefore, it is recommended that customers do a pilot evaluation using production data either on-premise or on AWS before arriving at an exact number of worker nodes


Browser Support and Hardware Requirements

$
0
0

Browser Support and Hardware Requirements

Supported Operating Systems and Browsers

Hardware Requirements for Supervisor and Worker Nodes

Hardware Requirements for Collector Nodes

Hardware Requirements for Report Server Nodes

Supported Operating Systems and Browsers

These are the browsers and operating systems that are supported for use with the FortiSIEM web client.

OS Supported Browsers Supported
 Windows Firefox, Chrome, Internet Explorer 11.x, Microsoft Edge
Mac OS X Firefox, Chrome, Safari
Linux Firefox, Chrome

 

Hardware Requirements for Supervisor and Worker Nodes

The FortiSIEM Virtual Appliance can be installed using either storage configured within the ESX server or NFS storage. See the topic Configuring NFS Server for more information on working with NFS storage.

Event Data Storage Requirements

The storage requirement shown in the Event Data Storage column is only for the eventdb data, but the /data partition also includes CMDB backups and queries. You should set the /data partition to a larger amount of storage to accommodate for this.

Encryption for Communication Between FortiSIEM Virtual Appliances

All communication between Collectors that are installed on-premises and FortiSIEM Supervisors and Workers is secured by TLS 1.2 encryption. Communications are managed by OpenSSL/Apache  HTTP Server/mod_ssl on the Supervisor/Worker side, and libcurl, using the NSS library for SSL, on the Collector side.The FortiSIEM Supervisor/Workers use RSA certificate with 2048 bits as default.

 

You can control the exact ciphers used for communications between virtual appliances by editing the SSLCipherSuite section in the file /etc/httpd/conf.d/ssl.conf on FortiSIEM Supervisors and Workers. You can test the ciphersuite for your Super or worker using the following nmap command:

nmap –script ssl-cert,ssl-enum-ciphers -p 443 <super_or_worker_fqdn>

Calculating Events per Second (EPS) and Exceeding the License Limit

AccelOps calculates the EPS for your system using a counter that records the total number of received events in a three minute time interval. Every second, a thread wakes up and checks the counter value. If the counter is less than 110% of the license limit (using the calculation 1.1 x EPS License x 180) , then AccelOps will continue to collect events. If you exceed 110% of your licensed EPS, events are dropped for the remainder of the three minute window, and an email notification is triggered. At the end of the three minute window the counter resets and resumes receiving events.

Overall EPS Quantity Host SW Processor Memory OS/App and CMDB Storage Event Data Storage

(1 year)

1,500 1 ESXi (4.0 or later preferred) 4 Core 3 GHz, 64 bit 16 GB

24 GB

(4.5.1+)

200GB (80GB OS/App, 60GB CMDB, 60G

B SVN)

3 TB
4,500 1 ESXi (4.0 or later preferred) 4 Core 3 GHz, 64 bit 16 GB

24 GB

(4.5.1+)

200GB (80GB OS/App, 60GB CMDB, 60G

B SVN)

8 TB
7,500 1 Super

1 Worker

ESXi (4.0 or later preferred) Super: 8 Core 3 GHz, 64 bit

Worker: 4 Core 3

GHz, 64 bit

Super: 24 GB Worker:

16 GB

Super: 200GB (80GB OS/App, 60GB CMDB, 60GB SVN)

Worker: 200GB (80GB OS/App)

12 TB
10,000 1 Super

1 Worker

ESXi (4.0 or later preferred) Super: 8 Core 3 GHz, 64 bit

Worker: 4 Core 3

GHz, 64 bit

Super: 24 GB Worker:

16 GB

Super: 200GB (80GB OS/App, 60GB CMDB, 60GB SVN)

Worker: 200GB (80GB OS/App)

17 TB
20,000 1 Super

3 Workers

ESXi (4.0 or later preferred) Super: 8 Core 3 GHz, 64 bit

Worker: 4 Core 3

GHz, 64 bit

Super: 24 GB Worker:

16 GB

Super: 200GB (80GB OS/App, 60GB CMDB, 60GB SVN)

Worker: 200GB (80GB OS/App)

34 TB
30,000 1 Super

5 Workers

ESXi (4.0 or later preferred) Super: 8 Core 3 GHz, 64 bit

Worker: 4 Core 3

GHz, 64 bit

Super: 24 GB Worker:

16 GB

Super: 200GB (80GB OS/App, 60GB CMDB, 60GB SVN)

Worker: 200GB (80GB OS/App)

50 TB
Higher than

30,000

Consult

FortiSIEM

         
Hardware Requirements for Collector Nodes
Component Quantity Host SW Processor Memory OS/App Storage
Collector 1 ESX 2 Core 2 GHz, 64 bit 4 GB 40 GB
Collector 1 Native Linux

Suggested Platform: Dell PowerEdge R210 Rack Server

2 Core, 64 bit 4GB 40 GB
Hardware Requirements for Report Server Nodes
Component Quantity Host

SW

Processor Memory OS/App Storage Reports Data Storage (1 year)
Report

Server

1 ESX 8 Core 3

GHz, 64 bit

16 GB 200GB (80GB OS/App, 60GB

CMDB, 60GB SVN)

See recommendations under Hardware Requirements for

Supervisor and Worker nodes

 

 

 

Information Prerequisites for All FortiSIEM Installations

You should have this information ready before you begin installing the FortiSIEM virtual appliance on ESX:

  1. The static IP address and subnet mask for your FortiSIEM virtual appliance.
  2. The IP address of NFS mount point and NFS share name if using NFS storage. See the topics Configuring NFS Storage for VMware ESX Server and Setting Up NFS Storage in AWS for more information.
  3. The FortiSIEM host name within your local DNS server.
  4. The VMWare ESX datastore location where the virtual appliance image will be stored if using ESX storage.

 

Hypervisor Installations

$
0
0

Hypervisor Installations

Topics in this section cover the instructions for importing the AccelOps disk image into specific hypervisors and configuring the AccelOps virtual appliance. See the topics under General Installation for information on installation tasks that are common to all hypervisors.

Installing in Amazon Web Services (AWS)

Determining the Storage Type for EventDB in AWS

Configuring Local Storage in AWS for EventDB

Setting Up Supervisor, Worker and Collector Nodes in AWS

Setting Up AWS Instances

Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS

Configuring the Supervisor and Worker Nodes in AWS

Registering the Collector to the Supervisor in AWS

Setting up a Network Bridge for Installing AccelOps in KVM

Importing the Supervisor, Collector, or Worker Image into KVM Configuring Supervisor Hardware Settings in KVM

Importing a Supervisor, Collector, or Worker Image into Microsoft Hyper-V

Setting the Network Time Protocol (NTP) for ESX

Installing a Supervisor, Worker, or Collector Node in ESX

Importing the Supervisor, Collector, or Worker Image into the ESX Server

Editing the Supervisor, Collector, or Worker Hardware Settings

Setting Local Storage for the Supervisor

Troubleshooting Tips for Supervisor Installations

Configuring the Supervisor, Worker, or Collector from the VM Console

Installing in Amazon Web Services (AWS)

You Must Use an Amazon Virtual Public Cloud with AccelOps

You must set up a Virtual Public Cloud (VPC) in Amazon Web Services for FortiSIEM deployment rather than classic-EC2. FortiSIEM does not support installation in classic-EC2. See the Amazon VPC documentation for more information on setting up and configuring a VPC. See Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS for information on how to prevent the public IPs of your instances from changing when they are stopped and started.

Using NFS Storage with Amazon Web Services

If the aggregate EPS for your FortiSIEM installation requires a cluster (an FortiSIEM virtual appliance + worker nodes), then you must set up an NFS server. If your storage requirements for the EventDB are more than 1TB, it is strongly recommended that you use an NFS server where you can configure LVM+RAID0. For more information, see Setting Up NFS Storage in AWS.

 

Determining the Storage Type for EventDB in AWS

Configuring Local Storage in AWS for EventDB

Setting Up Supervisor, Worker and Collector Nodes in AWS

Setting Up AWS Instances

Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS

Configuring the Supervisor and Worker Nodes in AWS

Registering the Collector to the Supervisor in AWS

Note: SVN password reset issue after system reboot for FortiSIEM 3.7.6 customers in AWS Virtual Private Cloud (VPC)

FortiSIEM uses SVN to store monitored device configurations. In AWS VPC setup, we have noticed that FortiSIEM SVN password gets changed if the system reboots – this prevents FortiSIEM from storing new configuration changes and viewing old configurations. The following procedure can be used to reset the SVN password to FortiSIEM factory default so that FortiSIEM can continue working correctly.

This script needs to be run only once.

  1. Logon to Super
  2. Copy the attached “ao_svnpwd_reset.sh” script to Super on EC2+VPC deployment
  3. Stop all backend processes before running script by issuing the following command: phtools –stop all
  4. Run following command to change script permissions: “chmod +x ao_svnpwd_reset.sh”
  5. Execute “ao_svnpwd_reset.sh” as root user: “./ao_svnpwd_reset.sh”
  6. The system will reboot
  7. Check SVN access to make sure that old configurations can be viewed.
Determining the Storage Type for EventDB in AWS

If the aggregate EPS for your FortiSIEM installation requires a cluster (a virtual appliance +  Worker nodes), then you must set up an NFS server as described in Using NFS Storage with Amazon Web Services. If your storage requirement for EventDB is more than 1TB, it is recommended that you use an NFS server where you can configure LVM+RAID0, which is also described in those topics. Although it is possible to set up a similar LVM+RAID0 on the FortiSIEM virtual appliance itself, this has not been tested.

Here’s an example of how to calculate storage requirements: At 5000 EPS, you can calculate daily storage requirements to be about 22-30GB (300k events take roughly 15-20MB on average in compressed format stored in eventDB). So, in order to have 6 months of data available for querying, you need to have 4 – 6TB of storage.

If you only need one FortiSIEM node and your storage requirements are lower than 1TB, and is not expected to ever grow beyond this limit, you can avoid setting up an NFS server and use a local EBS volume for EventDB. For this option, see the topic Configuring Local Storage in AWS for EventDB.

Configuring Local Storage in AWS for EventDB

Create the Local Storage Volume

Attach the Local Storage Volume to the Supervisor

Create the Local Storage Volume

  1. Log in to AWS.
  2. In the E2 dashboard, click Volumes.
  3. Click Create Volume.
  4. Set Size to 100 GB to 1 TB (depending on storage requirement).
  5. Select the same Availability Zone region as the FortiSIEM Supervisor instance.
  6. Click Create.

Attach the Local Storage Volume to the Supervisor

  1. In the EC2 dashboard, select the local storage volume.
  2. In the Actions menu, select Attach Volume.
  3. For Instance, enter the Supervisor ID.
  4. For Device, enter /dev/xvdi.
  5. Click Attach.

 

Setting Up Supervisor, Worker and Collector Nodes in AWS

The basic process for installing an FortiSIEM Supervisor, Worker, or Collector node is the same. Since Worker nodes are only used in deployments that use NFS storage, you should first configure your Supervisor node to use NFS storage, and then configure your Worker node using the Supervisor NFS mount point as the mount point for the Worker. See Configuring NFS Storage for VMware ESX Server for more information. Collector nodes are only used in multi-tenant deployments, and need to be registered with a running Supervisor node.

Setting Up AWS Instances

Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS

Configuring the Supervisor and Worker Nodes in AWS

Registering the Collector to the Supervisor in AWS

When you’re finished with the specific hypervisor setup process, you need to complete your installation by following the steps described under Ge neral Installation.

You Must Use an Amazon Virtual Public Cloud with AccelOps

You must set up a Virtual Public Cloud (VPC) in Amazon Web Services for FortiSIEM deployment rather than classic-EC2. FortiSIEM does not support installation in classic-EC2. See the Amazon VPC documentation for more information on setting up and configuring a VPC. See Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS for information on how to prevent the public IPs of your instances from changing when they are stopped and started.

Using NFS Storage with Amazon Web Services

If the aggregate EPS for your FortiSIEM installation requires a cluster (an FortiSIEM virtual appliance + worker nodes), then you must set up an NFS server. If your storage requirements for the EventDB are more than 1TB, it is strongly recommended that you use an NFS server where you can configure LVM+RAID0. For more information, see Setting Up NFS Storage in AWS.

Setting Up AWS Instances

You Must Use an Amazon Virtual Public Cloud with AccelOps

You must set up a Virtual Public Cloud (VPC) in Amazon Web Services for FortiSIEM deployment rather than classic-EC2. FortiSIEM does not support installation in classic-EC2. See the Amazon VPC documentation for more information on setting up and configuring a VPC. See Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS for information on how to prevent the public IPs of your instances from changing when they are stopped and started.

Using NFS Storage with Amazon Web Services

If the aggregate EPS for your FortiSIEM installation requires a cluster (an FortiSIEM virtual appliance + worker nodes), then you must set up an NFS server. If your storage requirements for the EventDB are more than 1TB, it is strongly recommended that you use an NFS server where you can configure LVM+RAID0. For more information, see Setting Up NFS Storage in AWS.

  1. Log in to your AWS account and navigate to the EC2 dashboard.
  2. Click Launch Instance.
  3. Click Community AMIs and search for the AMI ID associated with your version of FortiSIEM. The latest AMI IDs are on the image server where you download the other hypervisor images.
  4. Click Select.
  5. Click Compute Optimized.

Using C3 Instances

You should select one of the C3 instances with a Network Performance rating of High, or 10Gb performance. The current generation of C3 instances run on the latest Intel Xeons that AWS provides. If you are running these machines in production, it is significantly cheaper to use EC2 Reserved Instances (1 or 3 year) as opposed to on-demand instances.

  1. Click Next: Configure Instance Details.
  2. Review these configuration options:
Network and Subnet Select the VPC you set up for your instance.
Number of

Instances

For enterprise deployments, set to 1. For a configuration of 1 Supervisor + 2 Workers, set to 3. You can also add instances later to meet your needs.
Public IP Clear the option Automatically assign a public IP address to your instances if you want to use VPN.
Placement

Group

A placement group is a logical grouping for your cluster instances. Placement groups have low latency, full-bisection 10Gbps bandwidth between instances. Select an existing group or create a new one.
EBS

Optimized

Instance

An EBS optimized instance enables dedicated throughput between Amazon EBS and Amazon EC2, providing improved performance for your EBS volumes. Note that if you select this option, additional Amazon charges may apply.
  1. Click Next: Add Storage.
  2. For Size, Volume Type, and IOPS, set options for your configuration.
  3. Click Next: Tag Instance.
  4. Under Value, enter the Name you want to assign to all the instances you will launch, and then click Create Tag.

After you complete the launch process, you will have to rename each instance to correspond to its role in your configuration, such as

Supervisor, Worker1, Worker2.

  1. Click Next: Configure Security Group.
  2. Select Select an Existing Security Group, and then select the default security group for your VPC.

FortiSIEM needs access to HTTPS over port 443 for GUI and API access,  and access to SSH over port 22 for remote management, which are set in the default security group. This group will allow traffic between all instances within the VPC.

  1. Click Review and Launch.
  2. Review all your instance configuration information, and then click Launch.
  3. Select an existing or create a new Key Pair to connect to these instances via SSH.

If you use an existing key pair, make sure you have access to it. If you are creating a new key pair, download the private key and store it in a secure location accessible from the machine from where you usually connect to these AWS instances.

  1. Click Launch Instances.
  2. When the EC2 Dashboard reloads, check that all your instances are up and running.
  3. All your instances will be tagged with the Name you assigned in Step 11, select an instance to rename it according to its role in your deployment.
  4. For all types of instances, follow the instructions to SSH into the instances as described in Configuring the Supervisor and Worker Nodes in AWS, and then run the script sh to check the health of the instances.

Creating VPC-based Elastic IPs for Supervisor and Worker Nodes in AWS

You need to create VPC-based Elastic IPs and attach them to your nodes so the public IPs don’t change when you stop and start instances.

  1. Log in to the Amazon VPC Console.
  2. In the navigation pane, click Elastic IPs.
  3. Click Allocate New Address.
  4. In the Allocate New Address dialog box, in the Network platform list, select EC2-VPC, and then click Yes, Allocate.
  5. Select the Elastic IP address from the list, and then click Associate Address.
  6. In the Associate Address dialog box, select the network interface for the NAT instance. Select the address to associate the EIP with from the Private IP address list, and then click Yes, Associate.

Configuring the Supervisor and Worker Nodes in AWS

  1. From the EC2 dashboard, select the instance, and then click Connect.
  2. Select Connect with a standalone SSH client, and follow the instructions for connecting with an SSH client.

For the connection command, follow the example provided in the connection dialog, but substitute the FortiSIEM root user name for ec2user@xxxxxx. The ec2-user .name is used only for Amazon Linux NFS server.

  1. SSH to the Supervisor.
  2. Run cd /opt/phoenix/deployment/jumpbox/aws.
  3. Run the script pre-deployment.sh to configure host name and NFS mount point.
  4. Accept the License Agreements.
NFS Storage <NFS Server IP>:/data

For <NFS Server IP>, use the 10.0.0.X IP address of the NFS Server running within the VPC

Local Storage /dev/xvdi
  1. The system will reboot.
  2. Log in to the Supervisor.
  3. Register the Supervisor by following steps in
  4. Run cd /opt/phoenix/deployment/jumpbox/aws.
  5. Run the script sh (now includes running post-deployment.sh automatically).
  6. The system will reboot and is now ready.
  7. To install a worker node, follow steps 1-9 and the worker is ready
  8. To add a Worker to the cluster (assume Worker is already installed)
    1. Log in to the FortiSIEM GUI
    2. Go to Admin > License Management > VA Information
    3. Click Add
    4. Enter the private address of the Worker Node

Registering the Collector to the Supervisor in AWS

  1. Locate a Windows machine on AWS.
  2. Open a Remote desktop session from your PC to that Windows machine on AWS.
  3. Within the remote desktop session, launch a browser and navigate to https://<Collector-IP>:5480
  4. Enter the Collector setup information.
Name Collector Name
User ID Admin User
Password Admin Password
Cust/Org ID Organization Name
Cloud URL Supervisor URL
  1. Click

The Collector will restart automatically after registration succeeds.

FortiSIEM Installing in Linux KVM

$
0
0
Installing in Linux KVM

The basic process for installing an FortiSIEM Supervisor, Worker, or Collector node in Linux KVM is the same as installing these nodes under VMware ESX, and so you should follow the instructions in Installing a Supervisor, Worker, or Collector Node in ESX. Since Worker nodes are only used in deployments that use NFS storage, you should first configure your Supervisor node to use NFS storage, and then configure your Worker node using the Supervisor NFS mount point as the mount point for the Worker. Collector nodes are only used in multi-tenant deployments, and need to be registered with a running Supervisor node.

Setting up a Network Bridge for Installing AccelOps in KVM

Importing the Supervisor, Collector, or Worker Image into KVM Configuring Supervisor Hardware Settings in KVM

Setting up a Network Bridge for Installing AccelOps in KVM

If FortiSIEM is the first guest on KVM, then a bridge network may be required to enable network connectivity. For details see the KVM documentation provided by IBM.

In these instructions, br0 is the initial bridge network, em1 is connected as a management network, and em4 is connected to your local area network.

  1. In the KVM host, go to the directory /etc/sysconfig/network-scripts/.
  2. Create a bridge network config file ifcfg-br0.

 

DEVICE=br0

BOOTPROTO=none

NM_CONTROLLED=yes

ONBOOT=yes

TYPE=Bridge

NAME=”System br0″

  1. Edit network config file ifcfg-em4.

 

DEVICE=em4

BOOTPROTO=shared

NM_CONTROLLED=no

ONBOOT=yes

TYPE=Ethernet

UUID=”24078f8d-67f1-41d5-8eea-xxxxxxxxxxxx”

IPV6INIT=no

USERCTL=no

DEFROUTE=yes

IPV4_FAILURE_FATAL=yes

NAME=”System em4″

HWADDR=F0:4D:00:00:00:00 BRIDGE=br0

  1. Restart the network service.
Importing the Supervisor, Collector, or Worker Image into KVM
  1. Download and uncompress the FortiSIEM OVA package from the FortiSIEM image server to the location where you want to install the image.
  2. Start the KVM Virtual Machine Manager.
  3. Select and right-click on a host to open the Host Options menu, and then select New.
  4. In the New VM dialog, enter a Name for your FortiSIEM node.
  5. Select Import existing disk image, and then click Forward.
  6. Browse to the location of OVA package and select it.
  7. Choose the OS Type and Version you want to use with this installation, and then click Forward.
  8. Allocate Memory and CPUs to the FortiSIEM node as recommended in the topic Browser Support and Hardware Requirements, and then click Forward.
  9. Confirm the installation configuration of your node, and then click Finish.
Configuring Supervisor Hardware Settings in KVM
  1. In KVM Virtual Machine Manager, select the FortiSIEM Supervisor, and then click Open.
  2. Click the Information icon to view the Supervisor hardware settings.
  3. Select the Virtual Network Interface.
  4. For Source Device, select an available bridge network.

See Setting up a Network Bridge for Installing FortiSIEM in KVM for more information.

  1. For Device model, select Hypervisor default, and then click Apply.
  2. In the Supervisor Hardware settings, select Virtual Disk.
  3. In the Virtual Disk dialog, open the Advanced options, and for Disk bus, select IDE.
  4. Click Add Hardware, and then select Storage.
  5. Select the Select managed or other existing storage option, and then browse to the location for your storage.

You will want to set up a disk for both CMDB (60GB) and SVN (60GB). If you are setting up FortiSIEM Enterprise, you may also want to create a storage disk for EventDB, with Storage format set to Raw.

  1. In the KVM Virtual Machine Manager, connect to the FortiSIEM Supervisor and power it on.
  2. Follow the instructions in Configuring the Supervisor, Worker, or Collector from the VM Console to complete the installation.

Related Links

Configuring the Supervisor, Worker, or Collector from the VM Console

FortiSIEM Installing in Microsoft Hyper-V

$
0
0
Installing in Microsoft Hyper-V

These topics describe how to install FortiSIEM on a Microsoft Hyper-V virtual server.

Importing a Supervisor, Collector, or Worker Image into Microsoft Hyper-V

Supported Versions

FortiSIEM has been tested to run on Hyper-V on Microsoft Windows 2012.

 

Importing a Supervisor, Collector, or Worker Image into Microsoft Hyper-V

Using Local or NFS Storage for EventDB in Hyper-V

Before you install an FortiSIEM virtual appliance in Hyper-V, you should decide whether you plan to use NFS storage or local storage to store event information in EventDB. If you decide to use a local disk, you can add a data disk of appropriate size. Typically, this will be named as /dev/sdd if it is the 4th disk. When using local disk, choose the type ‘Dynamically expanding’ (VHDX) format so that you are able to resize the disk if your EventDB will grow beyond the initial capacity.

If you are going to use NFS storage for EventDB, follow the instructions in the topic Configuring NFS Storage for VMware ESX Server.

Disk Formats for Data Storage

FortiSIEM virtual appliances in Hyper-V use dynamically expanding VHD disks for the root and CMDB partitions, and a dynamically expanding VHDX disk for EventDB. Dynamically expanding disks are used to keep the exported Hyper-V image within reasonable limits. See the Microsoft documentation topic Performance Tuning Guidelines for Windows Server 2012 (or R2) for more information.

  1. Download and uncompress the the FortiSIEM OVA package from the FortiSIEM image server to the location where you want to install the image.
  2. Start Hyper-V Manager.
  3. In the Action menu, select Import Virtual Machine.

The Import Virtual Machine Wizard will launch.

  1. Click Next.
  2. Browse to the folder containing the OVA package, and then click Next.
  3. Select the FortiSIEM image, and then click Next.
  4. For Import Type, select Copy the virtual machine, and then click
  5. Select the storage folders for your virtual machine files, and then click Next.
  6. Select the storage folder for your virtual machine’s hard disks, and then click Next.
  7. Verify the installation configuration, and then click Finish.
  8. In Hyper-V Manager, connect to the FortiSIEM virtual appliance and power it on.
  9. Follow the instructions in Configuring the Supervisor, Worker, or Collector from the VM Console to complete the installation.

Related Links

Configuring the Supervisor, Worker, or Collector from the VM Console

FortiSIEM Installing in VMware ESX

$
0
0
Installing in VMware ESX

Setting the Network Time Protocol (NTP) for ESX

Installing a Supervisor, Worker, or Collector Node in ESX

Importing the Supervisor, Collector, or Worker Image into the ESX Server

Editing the Supervisor, Collector, or Worker Hardware Settings

Setting Local Storage for the Supervisor

Troubleshooting Tips for Supervisor Installations

Configuring the Supervisor, Worker, or Collector from the VM Console

Setting the Network Time Protocol (NTP) for ESX

It’s important that your Virtual Appliance has the accurate time in order to correlate events from multiple devices within the environment.

  1. Log in to your VMWare ESX server.
  2. Select your ESX host server.
  3. Click the Configuration
  4. Under Software, select Time Configuration.
  5. Click Properties.
  6. Select NTP Client Enabled.
  7. Click Options.
  8. Under General, select Start automatically.
  9. Under NTP Setting, click ...
  10. Enter the IP address of the NTP servers to use.

 

  1. Click Restart NTP service.
  2. Click OK to apply the changes.
Installing a Supervisor, Worker, or Collector Node in ESX

The basic process for installing an FortiSIEM Supervisor, Worker, or Collector node is the same. Since Worker nodes are only used in deployments that use NFS storage, you should first configure your Supervisor node to use NFS storage, and then configure your Worker node using the Supervisor NFS mount point as the mount point for the Worker. See Configuring NFS Storage for VMware ESX Server for more information. Collector nodes are only used in multi-tenant deployments, and need to be registered with a running Supervisor node.

Importing the Supervisor, Collector, or Worker Image into the ESX Server

Editing the Supervisor, Collector, or Worker Hardware Settings

Setting Local Storage for the Supervisor

Troubleshooting Tips for Supervisor Installations

When you’re finished with the specific hypervisor setup process, you need to complete your installation by following the steps described under Ge neral Installation.

 

 

 

 

Importing the Supervisor, Collector, or Worker Image into the ESX Server

  1. Download and uncompress the FortiSIEM OVA package from the FortiSIEM image server to the location where you want to install the image.
  2. Log in to the VMware vSphere Client.
  3. In the File menu, select Deploy OVF Template.
  4. Browse to the .ova file (example: FortiSIEM-VA-4.3.1.1145.ova) and select it.

On the OVF Details page you will see the product and file size information.

  1. Click Next.
  2. Click Accept to accept the “End User Licensing Agreement,” and then click Next.
  3. Enter a Name for the Supervisor or Worker, and then click Next.
  4. Select a Storage location for the installed file, and then click Next.

 

Running on VMWare ESX 6.0

If you are importing FortiSIEM VA, Collector, or Report Server images for VMWare on an ESXi 6.0 host, you will need to also “Upgrade VM Compatibility” to ESXi 6.0. If the VM is already started, you need to shutdown the VM, and use the “Actions” menu to do this. Due to some incompatibility created by VMWare, our collector VM processes restarted and the collector could not register with the supervisor. Similar problems are also likely to occur on supervisor, worker, or report server as well, so make sure their VM compatibilities are upgraded as well. More information about VM compatibility is available in the VMWare KB below:

https://kb.vmware.com/kb/1010675

Editing the Supervisor, Collector, or Worker Hardware Settings

Before you start the Supervisor, Worker, or Collector for the first time you need to make some changes to its hardware settings.

  1. In the VMware vSphere client, select the imported Supervisor, Worker, or Collector.
  2. Right-click on the node to open the Virtual Appliance Options menu, and then select Edit Settings… .
  3. Select the Hardware tab, and check that Memory is set to at least 16 GB and CPUs is set to 8.

Setting Local Storage for the Supervisor

Using NFS Storage

You can install the Supervisor using either native ESX storage or NFS storage. These instructions are for creating native EXS storage. See Configuring NFS Storage for VMware ESX Server for more information. If you are using NFS storage, you will set the IP address of the NFS server during Step 15 of the Configuring the Supervisor, Worker, or Collector from the VM Console process.

  1. On Hardware tab, click Add.
  2. In the Add Hardware dialog, select Hard Disk, and then click Next.
  3. Select Create a new virtual disk, and then click Next.
  4. Check that these selections are made in the Create a Disk dialog:
Disk Size 300GB

See the Hardware Requirements for Supervisor and Worker Nodes in the Browser Support and Hardware Requirements topic for more specific disk size recommendations based on Overall EPS.

Disk

Provisioning

Thick Provision Lazy Zeroed
Location Store to the Virtual Machine
  1. In the Advanced Options dialog, make sure that the Independent option for Mode is not selected.
  2. Check all the options for creating the virtual disk, and then click Finish.
  3. In the Virtual Machine Properties dialog, click OK. The Reconfigure virtual machine task will launch.

Troubleshooting Tips for Supervisor Installations

Check the  Supervisor System and Directory Level Permissions Check Backend System Health

Check the  Supervisor System and Directory Level Permissions

Use SSH to connect to the Supervisor and check that the cmdb, data, query, querywkr, and svn permissions match those in this table:

 

[root@super ~]# ls -l / dr-xr-xr-x.   2 root     root      4096 Oct 15 11:09 bin dr-xr-xr-x.   5 root     root      1024 Oct 15 14:50 boot drwxr-xr-x    4 postgres postgres  4096 Nov 10 18:59 cmdb drwxr-xr-x    9 admin    admin     4096 Nov 11 11:32 data drwxr-xr-x   15 root     root      3560 Nov 10 11:11 dev -rw-r–r–    1 root     root        34 Nov 11 12:09 dump.rdb drwxr-xr-x.  93 root     root     12288 Nov 11 12:12 etc drwxr-xr-x.   4 root     root      4096 Nov 10 11:08 home dr-xr-xr-x.  11 root     root      4096 Oct 15 11:13 lib dr-xr-xr-x.   9 root     root     12288 Nov 10 19:13 lib64 drwx——.   2 root     root     16384 Oct 15 14:46 lost+found drwxr-xr-x.   2 root     root      4096 Sep 23  2011 media drwxr-xr-x.   2 root     root      4096 Sep 23  2011 mnt drwxr-xr-x.  10 root     root      4096 Nov 10 09:37 opt drwxr-xr-x    2 root     root      4096 Nov 10 11:10 pbin dr-xr-xr-x  289 root     root         0 Nov 10 11:13 proc drwxr-xr-x    8 admin    admin     4096 Nov 11 00:37 query drwxr-xr-x    8 admin    admin     4096 Nov 10 18:58 querywkr dr-xr-x—.   7 root     root      4096 Nov 10 19:13 root dr-xr-xr-x.   2 root     root     12288 Oct 15 11:08 sbin drwxr-xr-x.   2 root     root      4096 Oct 15 14:47 selinux drwxr-xr-x.   2 root     root      4096 Sep 23  2011 srv drwxr-xr-x    4 apache   apache    4096 Nov 10 18:58 svn drwxr-xr-x   13 root     root         0 Nov 10 11:13 sys drwxrwxrwt.   9 root     root      4096 Nov 11 12:12 tmp drwxr-xr-x.  15 root     root      4096 Oct 15 14:58 usr drwxr-xr-x.  21 root     root      4096 Oct 15 11:01 var

 

Check that the /data , /cmdb, and /svn directory level permissions match those in this table:

 

[root@super ~]# ls -l /data drwxr-xr-x 3 root     root     4096 Nov 11 02:52 archive drwxr-xr-x 3 admin    admin    4096 Nov 11 12:01 cache drwxr-xr-x 2 postgres postgres 4096 Nov 10 18:46 cmdb drwxr-xr-x 2 admin    admin    4096 Nov 10 19:04 custParser drwxr-xr-x 5 admin    admin    4096 Nov 11 00:29 eventdb drwxr-xr-x 2 admin    admin    4096 Nov 10 19:04 jmxXml drwxr-xr-x 2 admin    admin    4096 Nov 11 11:33 mibXml

[root@super ~]# ls -l /cmdb drwx—— 14 postgres postgres  4096 Nov 10 11:08 data

[root@super ~]# ls -l /svn drwxr-xr-x 6 apache apache  4096 Nov 10 18:58 repos

 

Check Backend System Health

Use SSH to connect to the supervisor and run phstatus to see if the system status metrics match those in this table:

 

 

[root@super ~]# phstatus

Every 1.0s: /opt/phoenix/bin/phstatus.py

System uptime:  12:37:58 up 17:24,  1 user,  load average: 0.06, 0.01, 0.00

Tasks: 20 total, 0 running, 20 sleeping, 0 stopped, 0 zombie

Cpu(s): 8 cores, 0.6%us, 0.7%sy, 0.0%ni, 98.6%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 16333720k total, 5466488k used, 10867232k free, 139660k buffers

Swap: 6291448k total, 0k used, 6291448k free, 1528488k cached

PROCESS                  UPTIME         CPU%           VIRT_MEM       RES_MEM phParser                 12:00:34    0              1788m          280m phQueryMaster            12:00:34    0              944m           63m phRuleMaster             12:00:34    0              596m           85m phRuleWorker             12:00:34    0              1256m          252m phQueryWorker            12:00:34    0              1273m          246m phDataManager            12:00:34    0              1505m          303m phDiscover               12:00:34    0              383m           32m phReportWorker           12:00:34    0              1322m          88m phReportMaster           12:00:34    0              435m           38m phIpIdentityWorker       12:00:34    0              907m           47m phIpIdentityMaster       12:00:34    0              373m           26m phAgentManager           12:00:34    0              881m           200m phCheckpoint             12:00:34    0              98m            23m phPerfMonitor            12:00:34    0              700m           40m phReportLoader           12:00:34    0              630m           233m phMonitor                31:21       0              1120m          25m Apache                   17:23:23    0              260m           11m

Node.js                  17:20:54    0              656m           35m

AppSvr                   17:23:16    0              8183m          1344m

DBSvr                    17:23:34    0              448m           17m

 

 

Configuring the Supervisor, Worker, or Collector from the VM Console
  1. In the VMware vSphere client, select the Supervisor, Worker, or Collector virtual appliance 2. Right-click to open the Virtual Appliance Options menu, and then select Power > Power On.
  2. In the Virtual Appliance Options menu, select Open Console.
  3. In VM console, select Set Timezone and then press Enter.
  4. Select your Location, and then press Enter.
  5. Select your Country, and then press Enter.
  6. Select your Timezone, and then press Enter.
  7. Review your Timezone information, select 1, and then press Enter.
  8. When the Configuration screen reloads, select Login, and then press Enter.
  9. Enter the default login credentials.
Login root
Password ProspectHills
  1. Run the vami_config_net script to configure the network.

 

  1. When prompted, enter the the information for these network components to configure the Static IP address: IP Address, Netmask, Gate way, DNS Server(s).
  2. Enter the Host name, and then press Enter.
  3. For the Supervisor, set either the Local or NFS storage mount point.

For a Worker, use the same IP address of the NFS server you set for the Supervisor.

Supervisor Local storage /dev/sdd
NFS storage <NFS_Server_IP_Address>:/<Directory_Path>

 

After you set the mount point, the Supervisor will automatically reboot, and in 15 to 25 minutes the Supervisor will be successfully configured.

ISO Installation

These topics cover installation of FortiSIEM from an ISO under a native file system such as Linux, also known as installing “on bare metal.”  Installing a Collector on Bare Metal Hardware

Viewing all 2380 articles
Browse latest View live