With all the escalating hype surrounding AI I figured I shall explore the relevance of web technologies and roles that web developers play in the foreseable future. Despite all the glamour advertised over ChatGPT, Google Gemini and Microsoft CoPilot it is essential to recognise that its greatest single point of failure is its dependence on network connectivity.
So I have been experimenting with offline setups of LLMs and SLMs the past few months and had a mix of trial and errors. Glad to post that I have somewhat succeeded in deploying WebLLM models locally:
Link to demo: https://vite-webllm.onrender.com/
Note that the code repository itself can run on localhost and be fully functional offline. The demo is just deployed for convenience in case anyone is curious to test it out!