SME

Freelance Subject Matter Expert (SME) - Gold Set Prompt Writing About the RoleWe are looking for experienced Subject Matter Experts (SMEs) to support the development of high-quality Gold Set prompts, assessments, and evaluation content for Large Language Models (LLMs).The role involves using your academic and professional subject expertise to create, review, and validate questions that assess an AI model’s ability to understand complex academic material, reason accurately, interpret evidence, and provide well-supported responses. This is a freelance, project-based opportunity suitable for professionals, researchers, academics, PhD holders, postgraduate professionals, and experienced subject-matter specialists.Key ResponsibilitiesDevelop high-quality prompts and assessment questions based on academic and professional subject matter.Create multiple-choice, analytical reasoning, short-answer, long-answer, and evidence-editing questions.Develop questions that test reading comprehension, critical thinking, analytical reasoning, evidence interpretation, and synthesis.Prepare accurate reference answers, expected answers, and evaluation criteria/rubrics for LLM assessment.Review AI-generated responses for factual accuracy, reasoning quality, relevance, and adherence to source material.Identify unsupported conclusions, logical errors, overgeneralizations, and misinterpretations of evidence.Ensure questions are challenging, unambiguous, and appropriate for the intended academic level.Create questions that require the model to distinguish between evidence, interpretation, and unsupported claims.Validate assessment content for subject accuracy and consistency.Follow detailed project guidelines and quality standards while working independently.Provide feedback and revisions based on quality-review requirements. Required ExpertiseCandidates should have strong academic or professional expertise in one or more of the following domains and sub-domains: Engineering and TechnologyCivil EngineeringInformation TechnologyComputer EngineeringMaterials/Manufacturing EngineeringPhysical SciencesChemistryPhysicsMaterials ScienceArts, Humanities, and Social SciencesSociologyPsychologyHuman EcologyLinguisticsLibrary/Information ScienceMedical ScienceMedicineMedical TechnologyRadiologyClinical Laboratory MedicineLife ScienceBiologyBotanyBiochemistryPlant BiotechnologyCandidate ProfilePhD professionals only.Strong academic reading and comprehension skills.Excellent written English and ability to communicate complex concepts clearly.Strong analytical and critical-thinking abilities.Ability to distinguish evidence-based conclusions from assumptions or overgeneralizations.Experience with academic research, technical writing, assessment development, content creation, or subject-matter review is preferred.Familiarity with AI/LLM evaluation, prompt writing, AI-generated content review, or data annotation is an advantage.Strong attention to detail and ability to follow detailed instructions and evaluation guidelines.Ability to work independently and meet project deadlines.Example of WorkYou may be asked to review an academic study and create questions that assess:Academic reading comprehensionEvidence-based reasoningInterpretation of research findingsStatistical and methodological limitationsCritical analysisDistinguishing correlation, causation, and inferenceIdentifying claims that go beyond the available evidenceShort synthesis and evidence editingEngagementJob Type: Freelance / ContractWork Arrangement: RemoteEngagement: Project-basedSchedule: Flexible, based on project requirementsCompensation: Competitive hourly/project-based rate based on expertise and domain