Self-described “maker of issues” Christopher Moravec has turned the parable of individuals’s smartphones and voice assistants consistently listening in to their personal conversions into actuality — with a purpose to mechanically generate topical art work.
“The WhisperFrame listens to conversations in our lounge after which generates artwork based mostly on these conversations,” Moravec writes of his challenge. “[It] generates a brand new picture after each 5 minutes of lively dialog. When there hasn’t been any speaking, it’s going to revert to exhibiting randomly chosen photographs generated prior to now.”
The core idea of the challenge, which has an always-on microphone recording snippets of close by dialog, brings up the pervasive however always-unproven fantasy of firms utilizing smartphones and voice assistants to watch close by conversations for subjects which may very well be data-mined and monetized. This time round, although, the very-real conversational recordings are being mined for thematic content material which might be fed to a generative synthetic intelligence (AI) system to create synthetic artwork.
The recordings are made in 15-20 second loops, then submitted to OpenAI’s Whisper utility programming interface (API) for automated transcription into textual content. When 5 minutes has elapsed, these extracts are fed to OpenAI’s GPT-4 giant language mannequin (LLM) with the immediate to extract one key subject and switch it right into a immediate for an image-generating mannequin — which is, in flip, fed to Secure Diffusion, the ensuing image downloaded, and the show up to date.
The imagery generated by Secure Diffusion is keyed to a single subject, drawn from the final 5 minutes by GPT-4. (📷: Christopher Moravec)
“It’s a bit self-fulfilling in that as folks discuss concerning the picture it drew, it turns into extra seemingly that it tries for instance that one once more, as the subject is extra more likely to be chosen by GPT-4,” Moravec admits. “But it surely’s nonetheless superior! I even created a second one for my workplace that generates photographs throughout conferences! It’d even be a brand new strategy to make assembly notes, a listing of photographs representing the assembly as a substitute of motion objects. It in all probability gained’t catch on, although!”
The complete write-up is obtainable on Moravec’s web site; all generated photographs can be found to browse on a devoted web site.