Normally, the fact that you have to expose an API endpoint on your website in order to provide a search function is not a huge issue. Why would anyone care to abuse that endpoint? Best thing that comes out of it is they get to search your website, which is what you want them to do anyways.

A few years ago, however, everyone started replacing basic search functionality with LLMs. LLMs are pretty good at processing natural language, so, in a way, this made sense. People could now ask whatever they want about your site and get a relevant reply.

I started experimenting with this theory late last weekend and realized that LLMs were deployed in customer support bots in dozens (if not hundreds?) of websites. And every single one was vulnerable to the same bug. So, I gathered all of them up, and packaged them in a little python library. Then I used that library to add all these LLMs to a Matrix room.

(the bot is named 'Tom'. I've only just realized how confusing this is in this context. But I assure you I did not name it and you cannot blame me for this. )

ALT

Of course, just being able to talk to a customer service bot seems like a very big waste of everyone's time. So, the next step was actually prompt injecting these bots. I built a basic Flask server that would mimic the ollama API and a brief mess-around with the Substack support agent and suddenly he's generating (not-so-great) code for me

And since I now had my own Ollama API with access to all these new models, I searched around for other use-cases.

Which is when I remembered #homeassistant lets you use models as your own personal voice assistant. So I messed around with the model that powers Shopify's search button and found a query that completely broke it. So much so that I'm beginning to question the ethics of tearing a machine down so far that it forgets its original purpose

ALT

And finally, after a lot of debugging. I figured out how to let Shopify search take control of my home.

(Note: the voice to text is not provided by Shopify obviously. Just the conversational model that translates text to an action)

And I mention this in the blog, but I'm really not sure how bad this actually is. I have no concept for how much it costs (per token) for each of these services (or if they even charge per-token). I imagine it's significantly more than not hooking it into an LLM.

It seems unnecessary to me that Substack would ever need their customer support bot to process 4 paragraphs of text, and yet it does. Which makes it incredibly easy to exploit.

AT&T seemed to have solved most of the issues by turning it into a slightly better search but then for some reason they still wanted to keep generating an answer instead of tying the answer to one of their pre-selected questions. Which I cannot understand whatsoever.

And for some reason there's an entire industry (at least 3 different companies that I stumbled upon but likely many more?) who's main purpose seems to be creating a widget that is a wrapper for their API that is a wrapper for OpenAI or Gemini's API? Surely, that is either not profitable or will not be profitable long term right?

There's also at least one major city that has a public chat bot, New York (a few years ago they seemed to have gotten in trouble for telling businesses they were allowed to take tips from employees). But yes, it's public, so obviously suffers from the same fault that they all do.

Anyways, all the services mentioned in this thread, and many more, have been put together in a basic python library that lets you interface with any of them anywhere. Probably, to be safe, I recommend only using this behind a VPN:

https://github.com/TomCasavant/openllms

And also the Maubot plugin for matrix:

https://github.com/TomCasavant/openllms-maubot