【FinTech｜LLM 評測與安全核心】Senior AI Evaluation Engineer – TL
Job Description
公司介紹
我們是一家深耕金融科技與資料智能應用的技術團隊，正積極將 大型語言模型導入高風險、高標準的實際業務場景。不同於單純模型實驗，這裡更關注 AI 是否「可靠、可驗證、可長期運作」。
團隊由資深工程與資料背景成員組成，文化務實、重視工程品質，並持續投入 AI Safety、模型評測與自動化流程，打造能被真正信任的生成式 AI 系統。
工作內容
-
設計並開發 LLM / RAG 系統的自動化評測工具與流程
-
建立模型多維度評分機制（Accuracy、Relevance、Faithfulness、Consistency）
-
將評測流程整合至 CI/CD（Jenkins / GitLab CI），確保模型版本品質穩定
-
與業務及資料團隊合作，建立高品質 Golden Dataset 與 Synthetic Data
-
執行 Prompt Injection、Jailbreak 等 LLM 安全與紅隊測試
-
分析失敗案例，協助定位 Retrieval 或 Generation 問題，驗證修復成效
使用的技術
-
Python、Pytest / unittest
-
LLM / RAG Framework：LangChain、LlamaIndex、Semantic Kernel
-
資料分析：Pandas、NumPy、Visualization Tools
-
評測框架：RAGAS、TruLens、DeepEval、Promptfoo（加分）
-
CI/CD、Git
-
Docker / Kubernetes（加分）
Requirements
需求條件
- 計算機科學或相關科系學士以上
- 3 年以上 Python 開發經驗，熟悉測試與自動化流程
- 熟悉 NLP 與 LLM 應用實務
- 具備將技術結果轉化為結論與報告的能力
- 加分：FinTech 經驗、OWASP LLM Security、Kaggle / Open Source 貢獻
Interview process
若你/妳對本職缺內容感興趣，請準備履歷直接與 Cake 獵才招募顧問聯繫！
聯繫方式：
Consultant: Tina Lee
Mail: [email protected]
Line: https://line.me/ti/p/Qd-6Ql2Tc0
請先傳送履歷及詢問職缺的職缺連結或截圖給我，讓我快速了解你的背景。
我會盡快分享詳細公司資訊給您，介紹更多資訊及後續協助推薦職務 🙌🏻
About us
🔥This is the Cake Recruitment Consulting official web page🔥
Cake Recruitment Consulting - Executive Search, Contracting, EoR/Payroll
We offer a full range of services, including Executive Search, Contracting, and EoR/Payroll, to meet all your recruitment needs.
Our Special Advantages and Differentiation:
- Professional Recruiter + Direct Sourcing - All consultants have industry experience. In addition to providing accurate talent search. E.g Identify relevant client’s competitors, and engage in comprehensive talent acquisition, building long-term relationships with candidates, and providing market information to clients.
- CakeResume AI + Martech Recruitment - By leveraging digital Martech technology and employer branding, the recruitment process is optimized. Through a database of millions of potential candidates who would not normally appear, the number of candidates reached is increased and speeds up the sourcing process to fill the talent gap.
About Cake Recruitment Consulting
Cake is the Leading Asia talent platform. With over millions+ active resumes in our database and still it is increasing every day.
Currently more than 7,000+ clients are using our recruitment services.