A1111 for local image gen, better local text gen support
This update is mainly geared towards people who generate images or text locally using their NVIDIA GPU.
Overhauled the local image gen to use A1111 and its API. It generates images much faster than the original stable diffusion, and with less VRAM required!
Oobabooga API was not suitable for my text gen purposes because of the overhead in traditional event checks, so instead of going with that, I fixed some code in gpt_server to autoload the models better, so more models should be supported than before.
Image gen and text gen are also a little more transparent than before with the python code exposed under the "ai" directory, and being run via an included standalone python exe. This may also make it slightly easier to debug issues.
I also implemented preliminary support for Llama-based chat models, but this has only been briefly tested on my own computer, and only with the model Aitrepreneur/wizardLM-7B-GPTQ-4bit-128g which is small enough to fit on my GPU.
Overhauled the local image gen to use A1111 and its API. It generates images much faster than the original stable diffusion, and with less VRAM required!
Oobabooga API was not suitable for my text gen purposes because of the overhead in traditional event checks, so instead of going with that, I fixed some code in gpt_server to autoload the models better, so more models should be supported than before.
Image gen and text gen are also a little more transparent than before with the python code exposed under the "ai" directory, and being run via an included standalone python exe. This may also make it slightly easier to debug issues.
I also implemented preliminary support for Llama-based chat models, but this has only been briefly tested on my own computer, and only with the model Aitrepreneur/wizardLM-7B-GPTQ-4bit-128g which is small enough to fit on my GPU.