It might sound just like another contemporary Big Brother –scenario, but recent progress in identifying the location where a video was filmed, only by analyzing the images and sounds, could have some real benefits.
Spanish researchers from the La Salle campus at Ramón Llull University in Barcelona have developed a system to geo-locate videos with no indication of where they were produced on the map. The problem they are trying to solve is that millions of non geo-tagged videos available online are invisible to the eyes of geo-oriented applications. Such applications could for example be to locate missing persons and hostages. Although compared to initially low numbers, the researchers now have managed to improve the precision of results by a factor of four from earlier attempts, according to Kurzweil AI.
Many of the videos available online are accompanied by text which provides information on the place where it was filmed, but there is still a non-negligible chunk of videos that do not present this information. The approach used by the researchers in achieving their result allows the analyzed data to be purely audiovisual. All of the data obtained is merged together and grouped in clusters so that, using computer algorithms developed by the researchers, they can be compared with those of a collection of recorded videos already geo-located around the world, as reported by the Daily Mail. So, by cross-referencing the videos against the database of known footage the locations where the videos were filmed can be approximated. The research project deviates from previous attempts by placing a heavy weight on audio analysis together with the image comparison, as one of the researchers, Xavier Sevillano, tells Spanish tech site SINC:
“The acoustic information can be as valid as the visual and, on occasions, even more so when it comes to geo-locating a video. In this field we use some physics and mathematical vectors taken from the field of recognition of acoustic sources, because they have already demonstrated positive results”
In their study, published in the journal Information Sciences, the team has used almost 10,000 sequences as a reference from the MediaEval Placing Task audiovisual database, a benchmarking initiative or assessment of algorithms for multimedia content. The improvement with a factor of four as compared to earlier results, which was mentioned in the beginning of this article, regards previous trials with the same number of sequences from the same database. However, as also mentioned, the technology is in its infancy and the percentage of successful cases is still pretty low. The solution has proven that it is capable of locating 3% of videos within a ten-kilometer radius of their actual geographical location, and in 1% of cases it is accurate to one kilometer.
The researchers recognize that their method will require a much greater audiovisual base to apply it to the millions of videos that circulate on the internet, but they highlight its usefulness in locating those which do not have textual metadata and the potential possibilities that it offers. The team points out to SINC that the proposed system “despite having a limited database in terms of size and geographical coverage, is capable of geographically locating videos with more accuracy than its competitors”. If the researchers could work with larger data sets, the results would probably increase significantly.
One of the proposed applications of the technology is to geo-locate hostages in captivity. According to The Independent, since terror groups often film and release videos of hostages, the researchers have suggested that the technology could bring a new approach to locate the captured people. The newspaper quotes Xavier Sevillano:
“Our system does not make any assumptions regarding the location of the videos, but in these cases we are given very valuable additional information to limit the searches, as we already know that we are dealing with the area of Iraq or Syria, and therefore, we would only use reference videos from there”
However, even if counter-terrorist experts have welcomed the technology and called it interesting, they still see limitations today. For instance, because the technology partly relies on archive footage, it could be difficult to use it to geo-locate videos shot inside buildings or in remote areas where no archive footage exists. Fortunately for the researchers, they still think that the technology can contribute with real benefits to daily internet usage, so the sole aim is not only to fight terrorism and to locate other missing people who uploaded videos before their disappearance. Another much more immediate application could be to facilitate geographical browsing in video libraries, such as YouTube, according to Science Daily. Xavier Sevillano tells the website:
“For example, if I want to go on holiday to New York and I feel like watching videos of Manhattan, when I type in a search on YouTube I get videos coming up recorded on the island, but also the performance of the seventies group The Manhattans and the trailer of the Woody Allen film Manhattan, which are not relevant to my search, and in these cases, the new technology can also help”
Whatever applications the solution might find in the coming years, it shows the power of what technology might be able of in the future regarding making sense of data that are not put into text. As in many cases, the technology of geographically locating videos might not end up doing what the researchers had in mind when creating it, but become handy in fields of uses that we cannot predict today.