Category: Technology

Artificial Intelligence Thrives on Big Data Sets


For many years, science fiction movies have suggested that artificial intelligence would replicate all human thinking functions. While this may still be a long-term goal, that’s not what’s happening in the short-term. The focus of A.I. in the coming years will be to scan large data sets and deliver back actionable insights. It turns out that the form of this data is also important and impacts how well artificial intelligence can parse it.

Artificial Intelligence Can Plow Through Huge Data Sets

IBM, who has termed their form of artificial intelligence “Cognitive Computing”, aims to use machine intelligence to complement and augment, not replace, human function. IBM emphasizes deep learning and learning algorithms in its implementation of A.I to find patterns that humans can’t discern using manual tools. We often encounter scenarios in business situations where we have enough data but aren’t making the correct decisions when using that data. Artificial intelligence can not only parse these data sets but also help decision makers make informed choices based on the data. No one has the time to read thousands of pages of white papers or test results, which is exactly the point at which AI should step in and help humans. Machine algorithms and machine learning can go through reams of data and basically replace any human jobs that can be predictably performed.

Machine learning technologies like IBM’s have been deployed in a variety of big data scenarios:

  • Doctors will sometimes need to consult years of medical research data to try and find the best treatment modality for a particular disease. Thanks to increasingly powerful computers (and ever cheaper costs for processing cycles), doctors can rely on machine learning to parse medical abstracts to find the data they need to make informed decisions. This form of augmented intelligence represents cooperation between humans and machines whose partnership is more productive than each of the parts separately.
  • Social media produces a huge amount of data. The number of tweets is estimated to be approximately 500 million per day. That’s just coming from Twitter, never mind the other status updates from Facebook and the like. A.I. bots now regularly mine social networks for user data, with sentiment analysis being one of the most popular measures. For example, a politician might deploy an A.I. bot to gauge public sentiment after giving an important speech.
  • Humans generate lots of data during their daily interactions, all of which can be parsed for useful trends. Grocery stores can crawl through their trove of customer receipts to automatically shift purchasing towards more popular products and away from less popular foods. Anyone who owns a fitness tracker can use their own data to help themselves adopt healthier habits. A recent startup has even embedded A.I. into their fitness tracker to actively provide automated coaching to their customers.

Structured vs Unstructured Data

Supervised learning in artificially intelligent bots – where the A.I. bot works with data that has been pre-categorized – has matured to the point where researchers are now looking to take the next step: unsupervised learning. This will be an important advance since most data available to businesses and researchers is generally un-labeled data. This data has too much noise and is generally useless in supervised learning contexts. There are a number of companies who use human workers to categorize data into silos, from Amazon’s Mechanical Turk to CrowdFlower, but this is ultimately an added cost to businesses. The goal of A.I. practitioners is to deploy their bots on a given data input, without regard to massaging that data first, and getting back the desired output. A good example of this is automated machine translation, where any foreign text is translated to a target language without needing to clue the bot in on what the subject matter of the text is.

The bigger the models we are able to train, the better they will be. And you can’t have big models without lots of data, and most of that data is unstructured. If you’d like to learn more about how artificial intelligence can help your organization, contact us today.

    Leave Your Comment Here