RunanywhereAI/trnrt
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
Language:Python
Total stars: 1170
#python
#aws, #glm, #inference, #llm, #llmserving, #megakernels, #neuron, #nki, #openaiapi, #trainium
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
Language:Python
Total stars: 1170
#python
#aws, #glm, #inference, #llm, #llmserving, #megakernels, #neuron, #nki, #openaiapi, #trainium