[FEATURE] Add FOEM Support

[sankexin]

First-Order Error Matters: Accurate Compensation for Quantized Large Language Models.：https://arxiv.org/abs/2507.11017

https://github.com/Xingyu-Zheng/FOEM

FOEM+GPTAQ+SpinQuant will be better.

FOEM can be seamlessly integrated with advanced techniques such as GPTAQ and SpinQuant, yielding additional improvements under
the challenging W4A4KV4 setting, and further narrowing the accuracy gap with
full-precision baselines beyond what current state-of-the-art methods achieve.

[Qubitium]

@Xingyu-Zheng Would you be interested in helping to add FOEM quant + kernel support to GPTQModel?

[Xingyu-Zheng]

We are very glad to see that our FOEM has attracted attention! We believe we can complete the adaptation on GPTQModel within a few days, as the implementation of FOEM is quite simple.

However, unlike methods such as GPTQ and GPTAQ that have already been accepted at conferences, FOEM is still undergoing continuous improvements. We would like to ask whether the maintainers would prefer us to first integrate the current version into the repository and then continue maintaining it as updates are made? @Qubitium

[Qubitium]

@Xingyu-Zheng Awesome! No problem with the rolling updates and non-final status of the FOEM base code. I consider all software forever beta so no worries about stability. So yes, I would definitely recommend just merging what your team have into the current gptqmodel structure, confirm working status, create a small ci test to validate.

Unlike the original autogptq or gptq code, gptqmodel is structured like a pipeline with multiple serial stages. This allows the current code to adapt to many different quantization techniques in a modular way, i.e. Nvidia's EoRA. If you have MS Teams, feel free to msg me at qubitium@modelcloud.ai as I am always on teams for the most-part if you have any questions and issues during integration.