Unwitting Sensors: How DoD is Exploiting Social Media -
Consider a selection of tweets posted one day last month. Some were humorous, like those of Steven Colbert (@StephenAtHome) hawking his latest book: “For a free sample of my writing,” he quipped, “see this tweet.” Tiffany’s (@TiffanyAndCo) tweeted a costly tip: “Layer silver and gold pendants for an eclectic look.”
And supporters of Somalia’s al-Shabaab militant group used the Twitter handle @HSMPress to boast of three blasts in Mogadishu: “The third explosion occurred at the city’s former police station where the apostate militia were preparing to occupy.”
On any given day, 400 million short messages are typed or thumbed onto Twitter, and that’s just a fraction of the total communications sent through a social media universe that includes Facebook, Google+, chat rooms, bulletin boards and many other electronic forums. The messages are as diverse as all the conversations in the world, at least those parts where there are computers, smartphones and iPads. And the effort to extract important data points and patterns from this digital cacophony has become one of the biggest growth areas in intelligence.
“The face of ISR has changed,” says Joshua Hartman, a former senior Defense Department intelligence official who is CEO of Horizon Strategies Group. “Open-source and crowd-sourcing information is a critical component of ISR.”
The potential products of such tools are endless, and reach from the tactical and immediate to the strategic and global — not just to find out, say, who is “following” or retweeting the al-Shabaab postings, but to be able to query where anti-government sentiment may be clustered in Pakistan or what expat Iranians living in Dubai feel about the ayatollahs back home and at what time of day they feel that way. Is the riot in Cairo about food prices or about a video offensive to the prophet Mohammed?
Barry Costa, a senior program engineer at the federally funded MITRE Corp., recently coined the term “population-centric intelligence, surveillance and reconnaissance.” It’s gradually replacing another term MITRE came up with: “social radar.”
Costa says it’s not metaphor: “Sonar was invented back in the 1900s to let us see through the water. Radar was to see through the air. Infrared lets us see through the dark. And social radar or population-centric technologies let us see the human environment.”
Along the way, though, there is some unease, in privacy rights groups and even social media proponents, about the intelligence community exploiting these rich new streams of data: They may be open source, but the public communications of American citizens are all swept into the wash, sifted through and monitored by the technology.
Anyone can sign up for Twitter and follow a dozen or a hundred or a thousand fellow tweeters, but for a comprehensive analysis of all 400 million daily messages, you need to pay. What Twitter Inc. calls “the full Twitter Firehose” is a specific product that includes everything except private messages, and what the company calls “protected” tweets. Gold to marketing and brand analysts, the Firehose contains a stew of confessions, tastes and passions, a directory of people and their links to each other and what they are thinking about at any moment.
Among the small group of companies that buys the Firehose is Attensity Corp., a Palo Alto, Calif., company that sucks in Twitter and many other social media and then markets a vast amount of data and analysis.
“We have a massive-scale social media feed,” said Michelle de Haaff, Attensity’s vice president of strategy and corporate development. “150 million sources across 32 languages, and we sell the feed along with the analytics. And we sell that to the government and to commercial entities.”
Twitter’s Firehose is one of the stars in the Attensity showcase. “We have a contract directly with Twitter,” de Haaff said. “We get everything across every language. We are pulling the whole thing. We are able to sense, understand and find signals: sentiments, hot spots, trends, actions, intent.”
In countries where people use Twitter, she said, the company culls rich information.
“In Libya,” she said, “we were able to track everything: where the arms were, where the rebels were moving. We had on a map where everything was going.”
Among Attensity’s early and crucial investors was In-Q-Tel, the Central Intelligence Agency venture capital fund that backs technologies of potential use to intelligence agencies.
From the beginning, the CIA was not just an investor but also a customer.
“As part of their investment, the CIA gets the software. They get a license,” de Haaff said. “I can confirm they are still a customer, but that’s all I can do.”
Most of Attensity’s government sales are handled through a “reseller” called Inttensity, a social-media-analysis company based in Catonsville, Md. A paper on Inttensity’s website underlines the company’s “pre-negotiated access to the Twitter Firehose” and similar agreements “with social media aggregators for content to discussion boards, forums and blogs from around the world.”
In 2010, Inttensity won a Defense Intelligence Agency contract to provide a combination of the Inxight ThingFinder and Attensity Server text-extraction systems. (Inxight is another analytical system funded by the CIA’s investment arm.) That contract didn’t mention social media.
But two years later, the State Department awarded a $142,000 contract to Inttensity for a “social media command center,” according to the Federal Procurement Data System.
A source close to Inttensity said it’s an effort “to better understand foreign populations and what they really think.”
Last year, the company hosted a “federal social media summit” at the Pentagon City Ritz Carlton in Arlington, Va. Federal officials from various agencies packed the ballroom. One of the speakers was 30-year-old Dan Zarrella, a kind of social media scientist who had focused on commercial exploitation of the technology. He’d never realized the military and intelligence agencies were pursuing it as well.
“The thing that surprised me at the Inttensity conference is how much interest the federal government has and how far along they are in recognizing the power that’s there,” he said.
One of the efforts on this front has been driven by the Office of Naval Research, which doesn’t just fund MITRE’s social media research, but also coordinated a Lockheed Martin program to predict crises using traditional open-source media. An official from the office was a keynote speaker at the Inttensity event.
A Pentagon spokeswoman, U.S. Air Force Lt. Col. Melinda Morgan, declined an interview but emailed a response to questions about population-centric ISR. She said traditional ISR could detect a crowd’s activities but not a crowd’s intent.
“Population-centric ISR consists of technologies and techniques that allow us to see and understand the human environment,” she said. “Population ISR gives us a capability to determine narratives that drive behavior and allows us to more effectively communicate with audiences with whom we would engage. Population-centric ISR gives us a global and persistent indications and warnings capability that complements and enhances conventional sensors.”
At the GEOINT 2012 Symposium in Orlando, Fla., in October, several of the exhibitors were hawking social-media analytic products.
Among the companies was Aptima, which developed E-Meme software under a $750,000 contract with ONR’s Human Social, Cultural and Behavior Modeling Program. E-Meme stands for “Epidemiological Modeling of the Evolution of Messages,” and Aptima says it tracks the spread of ideas on the Internet in various countries. There’s an intriguing twist: “We are using the metaphor of it spreading like a disease,” explained Robert McCormack, associate director of Aptima’s analytics, modeling and simulation division. “We are looking at dynamics of transmission of ideas. For it to spread from person to person, you have to have some kind of contact between the people. And then you have to actually transmit the disease. Same thing here: The idea has to be transmitted.”
McCormack said the company doesn’t use Twitter at this point, just blogs and news sites. And he said it’s all still in the development stage.
Another company hoping to become a player in this realm is Booz Allen Hamilton, whose website claims the firm offers “the most powerful and sophisticated tools for extracting value from Social Data.”
At GEOINT, a program manager who asked that his name not be used showed off OSIRIS, one of the company’s social-media analysis programs, to display tweets published over the last 24 hours in Syria. “I look right here,” he said, using the mouse to move his cursor, “and we see a large spike take place. I can drill down into it and actually see the individual Twitter messages that took place.”
One tweet on the screen says “Smoke rising.” Another, heavily retweeted, message says, “three families have been wiped out in Hama, 54 people killed.”
Susan Kalweit, a principal in Booz Allen’s geospatial business, said the Twitter Firehose — which her company accesses via contract with DataSift, one of the big warehousers of this type of data in the commercial world — presents unique challenges. Much of Twitter’s content isn’t thumbed in by people, but generated by machines. And while U.S. tweets contain geographic location in their metadata, many foreign tweets don’t.
Kalweit said OSIRIS compensates for the lack of geographic metadata with software that recognizes place names and other hints about location, then attaches a latitude and longitude to the message. “We’ve been using Metacarta,” a geointelligence software system, “which we found is pretty good at getting definitely to the city level and often below the city level in some of these — not tourist — locations. So that’s really helpful when people talk about the city or about neighborhoods.”
Kalweit said social media presents new methods of obtaining intelligence from people who may see or comment on events in real time.
“I think that there is a construct of [the] human being as a sensor,” she said. “Human beings now are ground sensors.”
For all of its value to intelligence agencies, population-centric ISR raises privacy concerns. There is no “Firehose minus tweets from U.S. citizens.” Public tweets, no less than Internet postings of all sorts, are caught in the wash and monitored.
MITRE’s Costa points out that this is all public data that people posted to open forums. “We take active steps to ensure the privacy of the data. We buy publicly available data from public sellers,” he said. “When you look at Twitter, it is not one tweet that you care about; it is about 100 million tweets. It’s very rare you get to the individual Twitter level.”
Tweets, except the ones users have marked as private, are open for all to see. Indeed, the Library of Congress has said it plans to archive all public tweets for historians.
CIA spokesman Todd Ebitz emailed that “The CIA focuses exclusively on the collection of foreign intelligence. In fulfilling this mission, the agency is vigilant about the protection of American citizens’ civil liberties and privacy rights.”
And the Pentagon’s Morgan emailed that “all research is conducted strictly within the privacy guidelines set by law and DoD policies.” She added that the focus of population-centric ISR is overseas and that the programs “are designed to support Combatant Commanders with areas of operation outside the U.S.”
But even if each tweet and bulletin board posting is for public consumption, no more private then a scrawled message on a bathroom wall or graffiti on a bridge, there is unease about systematic government monitoring.
Rainey Reitman of the Electronic Frontier Foundation said that technically, it is indeed all legal, but she emphasized that people don’t really understand how their random thoughts, disclosures or opinions on social media may be exploited.
“I think people don’t realize when they sign up for these sites that the government is going to be routinely monitoring and sifting through this data,” she said.
“If Coca-Cola is reading all my tweets,” Dan Zarrella points out, “it’s not as scary as if the DOD is reading all my tweets, right?”
Read more -
http://www.defensenews.com/article/20121113/DEFREG02/311130003/Unwitting-Sensors-How-DoD-Exploiting-Social-Media
Consider a selection of tweets posted one day last month. Some were humorous, like those of Steven Colbert (@StephenAtHome) hawking his latest book: “For a free sample of my writing,” he quipped, “see this tweet.” Tiffany’s (@TiffanyAndCo) tweeted a costly tip: “Layer silver and gold pendants for an eclectic look.”
And supporters of Somalia’s al-Shabaab militant group used the Twitter handle @HSMPress to boast of three blasts in Mogadishu: “The third explosion occurred at the city’s former police station where the apostate militia were preparing to occupy.”
On any given day, 400 million short messages are typed or thumbed onto Twitter, and that’s just a fraction of the total communications sent through a social media universe that includes Facebook, Google+, chat rooms, bulletin boards and many other electronic forums. The messages are as diverse as all the conversations in the world, at least those parts where there are computers, smartphones and iPads. And the effort to extract important data points and patterns from this digital cacophony has become one of the biggest growth areas in intelligence.
“The face of ISR has changed,” says Joshua Hartman, a former senior Defense Department intelligence official who is CEO of Horizon Strategies Group. “Open-source and crowd-sourcing information is a critical component of ISR.”
The potential products of such tools are endless, and reach from the tactical and immediate to the strategic and global — not just to find out, say, who is “following” or retweeting the al-Shabaab postings, but to be able to query where anti-government sentiment may be clustered in Pakistan or what expat Iranians living in Dubai feel about the ayatollahs back home and at what time of day they feel that way. Is the riot in Cairo about food prices or about a video offensive to the prophet Mohammed?
Barry Costa, a senior program engineer at the federally funded MITRE Corp., recently coined the term “population-centric intelligence, surveillance and reconnaissance.” It’s gradually replacing another term MITRE came up with: “social radar.”
Costa says it’s not metaphor: “Sonar was invented back in the 1900s to let us see through the water. Radar was to see through the air. Infrared lets us see through the dark. And social radar or population-centric technologies let us see the human environment.”
Along the way, though, there is some unease, in privacy rights groups and even social media proponents, about the intelligence community exploiting these rich new streams of data: They may be open source, but the public communications of American citizens are all swept into the wash, sifted through and monitored by the technology.
Anyone can sign up for Twitter and follow a dozen or a hundred or a thousand fellow tweeters, but for a comprehensive analysis of all 400 million daily messages, you need to pay. What Twitter Inc. calls “the full Twitter Firehose” is a specific product that includes everything except private messages, and what the company calls “protected” tweets. Gold to marketing and brand analysts, the Firehose contains a stew of confessions, tastes and passions, a directory of people and their links to each other and what they are thinking about at any moment.
Among the small group of companies that buys the Firehose is Attensity Corp., a Palo Alto, Calif., company that sucks in Twitter and many other social media and then markets a vast amount of data and analysis.
“We have a massive-scale social media feed,” said Michelle de Haaff, Attensity’s vice president of strategy and corporate development. “150 million sources across 32 languages, and we sell the feed along with the analytics. And we sell that to the government and to commercial entities.”
Twitter’s Firehose is one of the stars in the Attensity showcase. “We have a contract directly with Twitter,” de Haaff said. “We get everything across every language. We are pulling the whole thing. We are able to sense, understand and find signals: sentiments, hot spots, trends, actions, intent.”
In countries where people use Twitter, she said, the company culls rich information.
“In Libya,” she said, “we were able to track everything: where the arms were, where the rebels were moving. We had on a map where everything was going.”
Among Attensity’s early and crucial investors was In-Q-Tel, the Central Intelligence Agency venture capital fund that backs technologies of potential use to intelligence agencies.
From the beginning, the CIA was not just an investor but also a customer.
“As part of their investment, the CIA gets the software. They get a license,” de Haaff said. “I can confirm they are still a customer, but that’s all I can do.”
Most of Attensity’s government sales are handled through a “reseller” called Inttensity, a social-media-analysis company based in Catonsville, Md. A paper on Inttensity’s website underlines the company’s “pre-negotiated access to the Twitter Firehose” and similar agreements “with social media aggregators for content to discussion boards, forums and blogs from around the world.”
In 2010, Inttensity won a Defense Intelligence Agency contract to provide a combination of the Inxight ThingFinder and Attensity Server text-extraction systems. (Inxight is another analytical system funded by the CIA’s investment arm.) That contract didn’t mention social media.
But two years later, the State Department awarded a $142,000 contract to Inttensity for a “social media command center,” according to the Federal Procurement Data System.
A source close to Inttensity said it’s an effort “to better understand foreign populations and what they really think.”
Last year, the company hosted a “federal social media summit” at the Pentagon City Ritz Carlton in Arlington, Va. Federal officials from various agencies packed the ballroom. One of the speakers was 30-year-old Dan Zarrella, a kind of social media scientist who had focused on commercial exploitation of the technology. He’d never realized the military and intelligence agencies were pursuing it as well.
“The thing that surprised me at the Inttensity conference is how much interest the federal government has and how far along they are in recognizing the power that’s there,” he said.
One of the efforts on this front has been driven by the Office of Naval Research, which doesn’t just fund MITRE’s social media research, but also coordinated a Lockheed Martin program to predict crises using traditional open-source media. An official from the office was a keynote speaker at the Inttensity event.
A Pentagon spokeswoman, U.S. Air Force Lt. Col. Melinda Morgan, declined an interview but emailed a response to questions about population-centric ISR. She said traditional ISR could detect a crowd’s activities but not a crowd’s intent.
“Population-centric ISR consists of technologies and techniques that allow us to see and understand the human environment,” she said. “Population ISR gives us a capability to determine narratives that drive behavior and allows us to more effectively communicate with audiences with whom we would engage. Population-centric ISR gives us a global and persistent indications and warnings capability that complements and enhances conventional sensors.”
At the GEOINT 2012 Symposium in Orlando, Fla., in October, several of the exhibitors were hawking social-media analytic products.
Among the companies was Aptima, which developed E-Meme software under a $750,000 contract with ONR’s Human Social, Cultural and Behavior Modeling Program. E-Meme stands for “Epidemiological Modeling of the Evolution of Messages,” and Aptima says it tracks the spread of ideas on the Internet in various countries. There’s an intriguing twist: “We are using the metaphor of it spreading like a disease,” explained Robert McCormack, associate director of Aptima’s analytics, modeling and simulation division. “We are looking at dynamics of transmission of ideas. For it to spread from person to person, you have to have some kind of contact between the people. And then you have to actually transmit the disease. Same thing here: The idea has to be transmitted.”
McCormack said the company doesn’t use Twitter at this point, just blogs and news sites. And he said it’s all still in the development stage.
Another company hoping to become a player in this realm is Booz Allen Hamilton, whose website claims the firm offers “the most powerful and sophisticated tools for extracting value from Social Data.”
At GEOINT, a program manager who asked that his name not be used showed off OSIRIS, one of the company’s social-media analysis programs, to display tweets published over the last 24 hours in Syria. “I look right here,” he said, using the mouse to move his cursor, “and we see a large spike take place. I can drill down into it and actually see the individual Twitter messages that took place.”
One tweet on the screen says “Smoke rising.” Another, heavily retweeted, message says, “three families have been wiped out in Hama, 54 people killed.”
Susan Kalweit, a principal in Booz Allen’s geospatial business, said the Twitter Firehose — which her company accesses via contract with DataSift, one of the big warehousers of this type of data in the commercial world — presents unique challenges. Much of Twitter’s content isn’t thumbed in by people, but generated by machines. And while U.S. tweets contain geographic location in their metadata, many foreign tweets don’t.
Kalweit said OSIRIS compensates for the lack of geographic metadata with software that recognizes place names and other hints about location, then attaches a latitude and longitude to the message. “We’ve been using Metacarta,” a geointelligence software system, “which we found is pretty good at getting definitely to the city level and often below the city level in some of these — not tourist — locations. So that’s really helpful when people talk about the city or about neighborhoods.”
Kalweit said social media presents new methods of obtaining intelligence from people who may see or comment on events in real time.
“I think that there is a construct of [the] human being as a sensor,” she said. “Human beings now are ground sensors.”
For all of its value to intelligence agencies, population-centric ISR raises privacy concerns. There is no “Firehose minus tweets from U.S. citizens.” Public tweets, no less than Internet postings of all sorts, are caught in the wash and monitored.
MITRE’s Costa points out that this is all public data that people posted to open forums. “We take active steps to ensure the privacy of the data. We buy publicly available data from public sellers,” he said. “When you look at Twitter, it is not one tweet that you care about; it is about 100 million tweets. It’s very rare you get to the individual Twitter level.”
Tweets, except the ones users have marked as private, are open for all to see. Indeed, the Library of Congress has said it plans to archive all public tweets for historians.
CIA spokesman Todd Ebitz emailed that “The CIA focuses exclusively on the collection of foreign intelligence. In fulfilling this mission, the agency is vigilant about the protection of American citizens’ civil liberties and privacy rights.”
And the Pentagon’s Morgan emailed that “all research is conducted strictly within the privacy guidelines set by law and DoD policies.” She added that the focus of population-centric ISR is overseas and that the programs “are designed to support Combatant Commanders with areas of operation outside the U.S.”
But even if each tweet and bulletin board posting is for public consumption, no more private then a scrawled message on a bathroom wall or graffiti on a bridge, there is unease about systematic government monitoring.
Rainey Reitman of the Electronic Frontier Foundation said that technically, it is indeed all legal, but she emphasized that people don’t really understand how their random thoughts, disclosures or opinions on social media may be exploited.
“I think people don’t realize when they sign up for these sites that the government is going to be routinely monitoring and sifting through this data,” she said.
“If Coca-Cola is reading all my tweets,” Dan Zarrella points out, “it’s not as scary as if the DOD is reading all my tweets, right?”
Read more -
http://www.defensenews.com/article/20121113/DEFREG02/311130003/Unwitting-Sensors-How-DoD-Exploiting-Social-Media
No comments:
Post a Comment