Annotation Guidelines
1. Your Role
As a Chinho annotator, you compare pairs of AI-generated Bangla responses and choose the one that better answers the prompt. Your judgments train AI systems to produce more accurate, natural, and helpful responses in Bangla.
2. How to Judge Responses
Consider these criteria in order of importance:
- Factual accuracy — Does the response contain correct information? Factual errors are the most serious problem.
- Completeness — Does the response fully address the question? Missing key details is a significant issue.
- Naturalness — Does the response sound natural in Bangla? Awkward phrasing or transliteration problems reduce quality.
- Helpfulness — Is the response practical and actionable for a Bangla-speaking user?
3. Writing Justifications
Every choice requires a brief justification in Bangla. Explain why the chosen response is better — not just that it is better. Reference specific differences between the two responses. One or two sentences is enough.
4. Voice Recording Tasks
For voice recording tasks, speak naturally in your own regional dialect. Do not use formal/standard Bangla unless that is your natural way of speaking. Record in a quiet environment, keep your microphone close, and stay within the 2-minute time limit.
5. Quality Standards
Your work is periodically checked against expert judgments (gold items). Maintaining a quality score above 70% is required to continue annotating. If your score drops, you will receive feedback on how to improve.
6. Payment
You earn a fixed amount per completed task, displayed before you begin each batch. Every task is reviewed. Payouts are sent via bKash or Nagad every Sunday, for work that has passed review. The minimum payout threshold is ৳100.
7. Code of Conduct
- Do not rush through tasks — quality matters more than speed.
- Do not share task content, screenshots or recordings outside Chinho: not on social media, Discord, messaging groups, websites or AI tools.
- Do not use automated tools or AI to complete your tasks.
- Report any issues or unclear prompts using the skip button with an appropriate reason.
- Do not copy, download or save task content, and never use it to train or test AI models or to build your own datasets.
- Task content stays confidential even after you stop working with Chinho. See section 6 of the Terms of Service.
8. Dialogue Writing
You write an original conversation between two people, set in everyday Bangladeshi life. It is paid once a reviewer approves it.
- Write it yourself, from your own life and what you have seen.
- Natural everyday speech, the way people really talk, not book language.
- Do not copy from the internet, books, films or AI tools.
- Do not write real people's names, phone numbers, addresses or other private details.
9. Scam Message Collection
You submit real scam and phishing messages, so that tools can learn to warn people about them. Each is paid once a reviewer approves it.
- Only real messages that you or people you know received.
- Before uploading, cover or blur private people's phone numbers and names, yours included. The scammer's number may stay.
- No made-up messages, and nothing copied from news or the internet.
- A message someone has already submitted is refused; send a different one.
10. Culture Questions
You write one four-option question about everyday life where you come from, and others answer it without seeing your answer. A question is paid once a reviewer approves it; an answer is paid when you submit it.
- Write from your own life and your own community.
- Do not take questions from exam guides, quiz books or the internet.
- No questions about politics or about named living people.
- Nothing that insults any group of people. When answering, say "I don't know" rather than guess.
11. Request Writing and Checking
You write one realistic request that someone in Bangladesh would type to an AI assistant, or check requests others wrote. A request is paid once a reviewer approves it; a check is paid when you submit it.
- Write it yourself. Do not copy from the internet or from AI tools.
- No real personal data: no names, phone numbers or addresses.
- Do not ask for anything harmful or illegal.
- A condition you add must be something anyone could check in an answer.