StableLM Releases New Open-Source Model, But Performance is Questionable: Is Transparency Enough?
StabilityAI, a company that specializes in language modelling, has recently released a new open-source model called stablelm-base-alpha-3b. However, there are concerns over the lack of benchmark results and performance evaluations of the model. The company has focused on marketing the model as transparent, accessible, and supportive, rather than on its actual performance. The StableLM model has been benchmarked against other open-source models using the MMLU benchmark, and the results were found to be underwhelming. Unfortunately, the company has not released any information about the benchmarks or details of the model. The fine-tuned model also does not seem to be working very well, and there are doubts about whether the license restrictions on training data affect the model itself.