Why did we build Talking Heads?

How do you get reliable information to people who prefer to get it on video, and in short pieces at a time? How do you distribute it in today’s decentralised social media environment?

These are the questions that drove the Talking Heads project. We wanted to see whether it was technically possible to create a new type of communication in the style of a video chatbot; whether this could be deployed on messaging networks, and whether people would be able to check the authenticity of content they received.

One of us, Andrei Nonchev of CoSoft, had already worked on something similar, the Histories of Belene project. That historical memory initiative consists of long interviews with  survivors of the Communist concentration camp in Belene, Bulgaria, in which people could use the system to access the survivors’ testimony through a chat interface.  Users would ask a question (such as: Why were you put in the camp? How did the guards treat you? What was the food like? etc.), and the system would find the best piece of pre-recorded testimony.

Because it was a historical memory project, and one that featured testimony from survivors, reliability was an absolute requirement for the Histories of Belene project. And because Talking Heads is an attempt to distribute reliable information by video, it shares the same requirement. In neither case can hallucinations, common in many high profile AI systems, including those built by major tech companies, be tolerated.

So we adapted Andrei’s design from Histories of Belene, which uses language models to select from a long list of pre-recorded videos, to this case of public policy communication.  This guarantees the answers the system produces are ones that were really said by the people we recorded.  Nevertheless by recording a large number of videos, our AI is able to select relevant answers (by performing a similarity search of embeddings), and allow a reasonably naturalistic conversation to take place.

We then added the ability to converse with our recordings using messaging servcies, like WhatsApp. This allows people to share the videos they receive in their personal and professional networks, potentially making this system a powerful way to distribute reliable information and giving everyone the ability to use them to counter rumours and misinformation.  Instead of trying to consult a fact checking website, which usually only appeal to a minority, people can ask for reliable information on an issue from the system we have developed, and share it as you see fit.
 

Education in Bulgaria

We chose the education system in Bulgaria as our topic because it is of broad, almost universal, interest in the country, but is one in which people often have limited knowledge that goes beyond their immediate personal experience.  We conducted long interviews with four people:

Iskra Djanabetska
Iskra is the founder of a platform that stimulates children's reading. Part of it is the children's media site "Vizhte", which explains the world in children's language. She can answer your questions related to education, children's reading, functional and media literacy.

Maya Kerezieva
Maya is  a parent of two students and she is one of the more active parent volunteers who willingly participate in school life. Maya has a lot to share in terms of relationships and cooperation between teachers and parents and the school administration.
 

Stefan Kondov
Stefan completed his secondary education in the small Bulgarian town of Bratsigovo. You can ask him about life and education in the small town and the challenges of transitioning to life in Sofia and starting a job.

Zoiko Milenkov
Zoiko graduated from Sofia University with a degree in Social Work. As a child, he grew up in a home for children deprived of parental care in the town of Lom. He can tell you about his life in the small town and his opinion on the education system through the eyes of the most vulnerable social groups in Bulgaria.

Each brings different perspectives on Bulgaria’s education system, as well as stories from their personal life and upbringing 

All four have material in Bulgarian, and have been given automatically generated English subtitle's.Iskra Djanabetska’s bot has been trained to respond to more idiomatic English so we recommend engaging with Iskra if you ask questions in English but you may attempt an English conversation with all characters.

Chat with them now either on this site or through WhatsApp at +1401 648 4194.

Lessons

Technical Implementation We set out to find out whether the web-based chat interface could work well on established messaging networks. This was a higher threshold to meet than just finding out whether the interface could work in the visual format supplied by a phone. But it remained to be seen whether the experience could work within the ilmits of messagin apps' own interfaces and at a speed that would make a continuous conversation possible. We were able to design a system that reacted quickly enough and alllows converations in multiple languages with multiple characters. The videos can then be shared within people's networks.

Going Forward. We however stuck too closely to the historical memory template, which envisions long interactions with a particulalr character. There sre certainly cases where this is what users will want, and this long-form version has applications for historical storytelling, such as

• Interactive biographies that recreate characters from biographies and archival research;
• Animating verbatim transcripts where such material is available, for example to dramatise the record of a public inquiry or parlilamentary debates;
• Fiction and entertainment. We could add interactivity so that the system can present choices to users and behave differentlly depending on the  option provided, alowing the creation of interactive storytelling games;

But there are other cases, involving the supply public information where the users don’t want that deep level of interaction; rather they would use this interface as a new kind of search engine to a public information service: they need specific reliable and authenticated information, quickly and audiovisually, but they would find engaging with a "character" representing, for instance, an urban parking permit service, tedious.

Alternatively, creators might want to accumulate content associated with a particular use case or character over time  in response to events, or use existing footage. We can adapt the system to  provide an interface for this kind of activity, and even create a platform where you can follow content creators but can access their content through natrual language converdation and algorithm that you control

Finally there’s an expert information case, useful for political communication, and information gathering. A politician could use it to canvass opinions in an area, or an expert have it as a gateay to provide technical advice, at scale, because the recodings only need to  be made once, but reliably and without hallucination,.

To bring these possiblities to life we’re creating the Mimesys.io platform.

The "Talking Heads" projects is funded by THE CIVICS Innovation hub.

“The implementation of the project “Talking Heads” was possible thanks to the Civic Innovation Fund (CIF), a unique European pooled fund of THE CIVICS Innovation Hub. CIF is supported by the King Baudouin Foundation”