The Business Intelligence Blog

Sensors everywhere. Infinite storage. Clouds of processors. Our ability to capture, warehouse, and understand massive amounts of data is changing science, medicine, business, and technology. As our collection of facts and figures grows, so will the opportunity to find answers to fundamental questions. Because in the era of big data, more isn’t just more. More is different.

This month’s Wired magazine carries one of the most important growing concerns of the scientific community, the uncontrollable growth of data. This growth of data in many directions is nearly killing theories as everything is becoming more and more data controlled.

There are a series of articles ranging from what data miners are digging today to elaborate algorithms that predict air ticket prices to how we can monitor epidemics hour by hour.

If you are a BI entusiast or not, this month’s Wired cover story will challenge all your predictions about science and technology, even if you have a petabyte of data to support it !! Read it, like, right now !!

From IT Web Article -

Business intelligence firm SAS is set to release a solution that enables companies to measure how their operations impact the environment – a tough sell for now but a step in the right direction, according to one analyst, says Inquirer.net.

SAS says its Sustainability Management solution allows end-users to measure business based on what it refers to as the “triple bottom line” of environmental, social and economic indicators.

SAS says its solution uses predefined “green” metrics from the Global Reporting Initiative, used by more than thousands of businesses to measure the environmental impact of their operations.

From Silicon Republic, an interesting article on the exponential growth of data -

CURRENTLY the amount of data worldwide is doubling every 11 months – by 2010 it will double every 11 hours.

Ireland stands in the direct path of this tidal wave of data, warns a senior executive with business intelligence (BI) giant SAS.

Dr John Brocklebank, director of analytic solutions at privately held software giant SAS Institute, believes Irish firms are ill-equipped to deal with this rapid growth in the world’s data.

He says the challenge for Irish companies is to capture and exploit the 1-2pc of data that is relevant to their decision-making processes and strategic objectives.

“We know that, on average, managers spend two hours a day looking for data and that more than half of this is useless to their decision-making process,” Brocklebank explains.

“Most frightening is that 42pc of managers say they accidentally use the wrong information to make a decision at least once a week. So never mind trying to deal with the data in its entirety; it needs to be made meaningful and accurate to support business decisions.

Business Objects is now officially into Predictive Analytics with its new tool Predictive Workbench and that’s a good news. Read more from Infoweek.

Business Objects on Wednesday announced Predictive Workbench, a new module for its business intelligence platform that lets businesses make predictions about such things as customer behavior and business performance. The module, based on technology from SPSS Inc., is the latest example of a traditional BI company looking to move beyond its expertise in reporting and analysis tools.

Predictive Workbench is the result of an OEM deal Business Objects struck last December to offer SPSS’s technology. IBM (NYSE: IBM)’s Cognos struck a similar deal with SPSS in March, and also plans to integrate IBM-developed predictive analytics within Cognos. Both efforts take a direct competitive shot at SAS Institute, by far the market leader in predictive analytics.

..
..

Since the December OEM deal, Business Objects has worked to integrate the SPSS technology, called Clementine, into Business Objects X1 3.0 so that it’s “totally transparent and seamless for customers,” said Franz Aman, VP of BI product marketing. The result is Predictive Workbench, from which a user can launch the Business Objects Universe metadata layer to run predictive analytics against various databases and data warehouses. Business Objects will disclose the price of the module only to interested customers, Aman says.

CRM Buyer has a series on breakthrughs in analytics world. A good read. Part 1 and Part 2.

While BI is about quickly obtaining enterprise information, Web analytics encompasses the collection, analysis and reporting of information about user activity on company Web sites. This process of analyzing the behavior of Internet visitors involves the study of the impact of a Web site on its users.

Distinctions between the two should be disappearing fairly quickly, according to Gary Angel, president and CTO of Semphonic, a tool-independent Web and search engine marketing analytics consultancy.

“I think it’s clear that there’s an increasing merger between traditional BI and Web analytics,” wrote Angel on his SemAngel blog. “This is true both in terms of data integration and tools. That’s certainly going to accelerate, and I see no reason why, in three years, the two disciplines will be separate in any meaningful sense. In addition, I think we’ll start to see much more ‘data-driven’ analysis within Web analytics.”

In 2006, IDC estimated the size of the 2005 Web analytics market at US$318 million and projected it to more than double in the ensuing five years. JupiterResearch put the size of the Web analytics market at $463 million in 2006. Today, the market is above $500 million.

This growth has occurred because the Web has become a part of the marketing mix model and is proving its value, according to Jim Sterne, president of the Web Analytics Association (WAA).

“Today, Web analytics tools can do their magic from afar,” Sterne told TechNewsWorld. “Ten years ago, we weren’t thinking about selling Software as a Service. Today’s tools are also much more capable of capturing the growing quantity of data and segmenting visitors to ensure the best possible response to a click. Further, 21st century tools are becoming more integrated with other marketing systems like e-mail, direct mail, telemarketing and in-store sales.”

E-commerce companies often use Web analytics software to measure such concrete details as how many people visited their site, how many of those visitors were unique visitors, how they came to the site (i.e., if they followed a link to get to the site or came there directly), what keywords they searched with on the site’s search engine, how long they stayed on a given page or on the entire site, what links they clicked on and when they left the site. Web analytic software can also be used to monitor whether or not a site’s pages are working properly.

With this information, Web site administrators can determine which areas of the site are popular and which areas of the site do not get traffic. They can then use this data to streamline a site to create a better user experience.

Read more from the Computer World article -

Based on the open-source Gecko browser engine from Mozilla, Strata is designed for people who need to create ad hoc reports from myriad data sources. It has its own scripting language so you can write a script that, say, monitors a Web site for changes to your data. Kirix also produces packaged scripts, or extensions. For example, by September, it will offer one that encrypts data going in and out of Strata.

Currently available for Windows and Linux users (a Mac version is ready but not yet shipping), Strata sells for $249 per seat.

One Petabyte of mixed relational and unstructure data. That’s neat.

More from the Sybase Press Release -

Sybase, Inc. (NYSE: SY), the largest enterprise software and services company exclusively focused on managing and mobilizing information, today announced that the Sybase® IQ analytics server has set a new Guinness World Record™ by powering the world’s largest data warehouse on a Sun™ SPARC® Enterprise M9000 server. This accomplishment was achieved using Sybase IQ, BMMsoft ServerSM and the Sun Microsystems® Data Warehouse Reference Architecture. This winning combination enables more data to be stored in less space, searched and analyzed in less time, while consuming 91 percent less energy and generating less heat and carbon dioxide than conventional solutions.

Powered by the category-leading column-oriented database Sybase IQ, the data warehouse is certified to support a record-breaking one petabyte of mixed relational and unstructured data—more than 34 times larger than the largest industry standard benchmark1 and twice the size of the largest commercial data warehouse known to date2. In total, the data warehouse contains six trillion rows of transactional data and more than 185 million content-searchable documents, such as emails, reports, spreadsheets and other multimedia objects.
..
..
Designed from the ground up as an analytics server, Sybase IQ produces its impressive results because of a unique architecture combining a column-oriented data structure with patented indexing and a scalable grid. Sybase IQ offers extraordinarily high performance at a lower cost than a traditional, row-oriented database. And, unlike traditional row-based data warehouses, the stored data in Sybase IQ is compressed by up to 70 percent of its input size, creating the most optimal and elegant analytics solutions.

“The results of this benchmark showcase Sybase IQ’s capabilities to handle real-world scenarios, querying vast amounts of data representing the transactions processed across the worldwide financial trading networks over multiple years.” said Francois Raab, president, InfoSizing, the consulting firm that oversaw the benchmarking of the record. “Sybase IQ has proven its production strength in handling the volume of multimedia documents representative of the electronic communication between half a million financial traders.”

Seth Grimes writes a very interesting article in the B-Eye Network, on Text Analytics and how accurate they are in deployments.

A Must Read for Text Analytics Teams.

Here’s an interesting paragraph from the article -

The accuracy of information retrieval (for instance, the results returned by a search) and of information extraction (where important entities, concepts and facts are pulled from “unstructured” sources) is typically measured by an f-score, a value based on two factors – precision and recall.

Precision is the proportion of information found that is correct or relevant. For example, if a Web search on “John Lennon” turns up 17 documents on Lennon and also 3 exclusively about Yoko Ono, who is of little interest but was associated with Lennon due to co-occurrence of the two individuals’ names in a large number of documents, then the precision proportion would be 17/20 or 85%.

Recall, by contrast, is the proportion of information found of information available. If there were actually 8 documents legitimately about John Lennon that were not found, perhaps because only a small portion of each was devoted to Lennon, leading to low “term density,” then the recall would be 17/25 or 68%.

According to Ian Michiels, an analyst with Aberdeen Group Inc., successful companies find a way to more tightly integrate marketing and sales. “How do you prioritize the leads that go into the sales pipeline? Metrics that help you do that are going to make you more effective,” he said. The goal is to find ways to define the best leads, whether that’s by the number of times a potential customer has gone to a Web site, attended a webinar or visited a trade-show booth. Only send the best leads to sales.

Once you provide this information, Rego said, your reps have the information they need to close more sales. “The carrot is providing sales with as much ammunition as needed to close the sale. The way you need [to] do that is tight integration from the marketing automation system into the SFA system.” Once you do this, you begin to build a consensus around the definition of a “good lead.” Rego said this approach gives marketing a view into the sales funnel and can provide marketing-level information as the sales staff approaches a sales opportunity.

Read the entire article on Marketing analytics at Inside CRM.

Varolii Corporation today announced Varolii Predictive Analytics(TM), an on-demand service that helps organizations understand, analyze and strategically target more effective customer outreach for collections, customer service and loyalty programs. Varolii unveiled Predictive Analytics at Interaction ‘08, the company’s inaugural customer conference.

Combining client data with the behavioral insight gleaned from billions of Varolii-led customer interactions, Varolii Predictive Analytics helps companies identify who to contact, when to contact them, through what channel, using which treatment, and how frequently in order to generate the greatest response. This combined data offers much more than a simple success or failure measurement; it’s a complete record of customer interaction and response on an individual level. The application is designed to gain intelligence from these conversations and progressively “learn” to predict customer behavior.
..
..
At the highest level, Varolii Predictive Analytics can help companies analyze contact strategy options (e.g., automated communications, an in-house call center, outsourced agents, or a combination of them) and select which will work best to solve their specific needs. At a more tactical level, it can help decide such things as an appropriate collection strategy based on Varolii and customer history, including probability to pay, days delinquent, and other metrics for improved results.

Read more from the Press Release.