Main page > Products > Semantic Archive > Monitoring and collecting data

Monitoring and collecting data

Monitoring and aggregation of data from the Internet is an important part of business intelligence. Analytical business solutions has profound expertise and experience in the field of automatic monitoring of sites and extracting data from open sources (websites and databases).

Monitoring and MBF platform are the two modules which are responsible for automated data aggregation in the Semantic Archive system. MBF platform is a new development which is being improved recently.


Monitoring module

This module is an in-house development of the company and is aimed at automatic capture of news from preset websites (BBC, CNN, Forbes and any other newslines, blogs, message boards, Twitter, etc.) and search engines such as Yandex, Google, Rambler, etc.

Monitoring has a handy interface and is easy to master. Visual editor is easy to use and will assist in customizing robots even without the knowledge of HTML.
This module fits best for simple media and news sites. It can be delivered with preset templates for both most popular and well-known Internet resources and industry-specific sites, a list of which is regularly extended (the list of sites can be obtained from ourSales Department.

The Monitoring module can operate independently from the Semantic Archive system. It can be used as an aggregator of news, for efficient viewing and reading news from selected sites, blogs and social networks at one place.

Monitoring main features:
  • Setting automated news aggregation from news sites, companies’ sites, social networks etc.;
  • Aggregation of news using keywords;
  • Aggregation of news with the use of RSS channels as well as without them;
  • Automatic extraction of document properties (source, URL, authors etc.);
  • Exchange of preset queries for news sites or search engines between users;
  • Automatic launch of monitoring tasks on a schedule;
  • Automatic launch of several tasks on different Internet sources simultaneously;
  • Use of filters and keywords for news selection. Highlighting sought-for combinations directly in the module;
  • Saving results as files to a disc, an SQL base or to the Semantic Archive database.

MBF Platform

This platform is an in-house development of the company and is aimed at data capture not only from news sites, but also blogs, message boards, search engines, social networks, online databases and providers of any content.

The unique feature of this technology is that the search algorithm of each robot is written for every particular source. This helps to reach maximum flexibility in processing, downloading, analyzing and storing data in the Semantic Archive system.

A settings and status panel is developed specifically for each robot. It allows users to set properties, which will be used by robots while operating. Robots can be launched both from a special module and directly from the system.


MBF main features:
  • Automatic data extraction from web sites and storing them in a convenient format in the Semantic Archive system. Parsing sites, tables, message boards, documents on sites;
  • Keeping record of all queries. You can always trace users history, what he searched for and which results did the particular search give;
  • Great variety of adjustable settings for download: topics, message boards, wall posts, comments, users accounts, photos, etc;
  • Generating and keeping a structured database of messages and records on users of social networks;
  • Flexible adjustments for search of negative and positive references, phrases, search objects;
  • Emulation of real users actions in search boxes of on-line bases and importing search results to the Semantic Archive database;
  • Search for objects of interest (companies or persons) taking into account synonyms and abbreviations on a big number of sites, online bases, search engines, social networks simultaneously and in real time;
  • Ability to choose between search results. The results are shown as a table with properties, where a user can easily select objects to be downloaded to the systems database;
  • Automatic anti-captcha;
  • Use of both external (Internet) and internal (any companys databases) robots;
  • And many more.

A Single Window function has been developed especially for economic security tasks. It allows to search for data in tax-free state online bases (Federal Tax Service, Supreme Court of Arbitration, public procurement, bankruptcies, negligent suppliers, Yandex blogs, Google blogs, social networks, etc.). A user can check a company or a person using several sources at once, including paid resources such as SPARK (www.spark-interfax.ru), Multistat, Official Single Register of Legal Entities.

Recently developed robots can automatically check companies and people in online bases of the following countries: Kazakhstan, Ukraine, Great Britain, USA and Germany. Full list and further details can be obtained from our Sales Department.

Site development SeaDesign O2