diff --git a/.gitignore b/.gitignore index 23c550a..87af2cb 100644 --- a/.gitignore +++ b/.gitignore @@ -40,3 +40,5 @@ __pycache__/ # Matrix metadata **/.omc/ + +data/vector-store.json diff --git a/BASE-README.md b/BASE-README.md new file mode 100644 index 0000000..17b9d19 --- /dev/null +++ b/BASE-README.md @@ -0,0 +1,92 @@ +# Stage 1: 고객지원 챗봇 만들기 + +Spring AI로 FAQ 챗봇을 직접 만드는 실습입니다. + +커리큘럼 소개는 [SYLLABUS.md](SYLLABUS.md)를, 진행 방법은 [GUIDE.md](GUIDE.md)를 참고하세요. + +--- + +## 필요한 것 + +| 항목 | 비고 | +|------|------| +| Java 17+ | `java -version`으로 확인 | +| OpenAI API 키 | [platform.openai.com](https://platform.openai.com)에서 발급 | +| IDE | IntelliJ IDEA 권장 (VS Code + Java Extension Pack도 가능) | + +- Spring Boot로 REST API를 만들어 본 경험이 있으면 됩니다 +- AI/ML 사전 지식은 없어도 됩니다 +- 끝까지 해도 API 비용은 **$1-5** 안쪽입니다 (GPT-4.1-nano) + +--- + +## 빠른 시작 + +```bash +# 프로젝트 루트에서 +cp .env.example .env # OpenAI API 키 입력 +./gradlew bootRun +``` + +서버가 `http://localhost:8080`에서 시작됩니다. + +그 다음 [mission/MISSION.md](mission/MISSION.md)를 여세요. 거기서부터 시작입니다. + +--- + +## 평가 + +`data/test_questions.json`에 100개의 테스트 질문이 있습니다 (easy 30 / medium 44 / hard 26). + +서버를 띄운 상태에서 평가 스크립트를 실행할 수 있습니다: + +```bash +cd data + +# Python 환경 준비 +python -m venv .venv +.venv/bin/pip install openai qdrant-client python-dotenv + +# 평가 실행 (judge 모델 gpt-4o-mini 사용, 100문항 기준 약 $0.5~1 추가 비용) +.venv/bin/python evaluate_rag.py +``` + +--- + +## 프로젝트 구조 + +``` +spring-ai-bootcamp-basic/ +├── mission/ +│ ├── MISSION.md # 미션 설명 (여기서 시작) +│ └── wall-report.md # 벽 리포트 (마지막에 작성) +├── hints/ +│ ├── HINT_01.md ~ HINT_06.md # 막혔을 때 열어보세요 +├── data/ +│ ├── layer1_faq/ # 공식 FAQ 문서 +│ ├── layer2_policies/ # 사내 정책 문서 +│ ├── layer3_chatlogs/ # 고객 상담 로그 +│ └── test_questions.json # 평가용 질문 100개 +├── src/ +│ └── main/java/com/cholog/bootcamp/ +│ └── Application.java # 여기서부터 만드세요 +├── SYLLABUS.md # 커리큘럼 소개 ← 과정 전체 그림 +├── GUIDE.md # 진행 가이드 ← 미션 중 참고 +└── build.gradle +``` + +--- + +## 자주 묻는 질문 + +**Q: API 키 없이 시작할 수 있나요?** + +코드 작성과 컴파일은 가능하지만, 실제 실행에는 OpenAI API 키가 필요합니다. + +**Q: FAQ 데이터가 영어인데 질문은 한국어로 해도 되나요?** + +네. GPT-4.1-nano는 교차 언어 이해가 가능합니다. + +**Q: 어떤 도구를 써도 되나요?** + +네. 구현 방법에 제약은 없습니다. diff --git a/KPI.md b/KPI.md new file mode 100644 index 0000000..d4bef68 --- /dev/null +++ b/KPI.md @@ -0,0 +1,43 @@ +# 챗봇 KPI 평가 영역 + +### 전체 +- 사용자 만족도 +- 비용(토큰 사용량) +- 챗봇 사용률/상담사 상담 감소율 +- 응답 속도 +- 사실 정확성 +- 답변 보안성 +- 챗봇 보안성 +- 답변 명확성 +- 챗봇 로그 관리 +- 모니터링 및 장애 대응 + +### 개발/품질 +- 응답 속도 +- 사실 정확성 +- 맥락 적합성 +- 보안성 +- 비용(토큰 사용량) +- 답변 명확성 + + +### 운영 +- 사용자 만족도 +- 챗봇 사용률/상담사 상담 감소율 +- 챗봇 로그 관리 +- 모니터링 및 장애 대응 +- 챗봇 보안성 + + +# 영역 별 평가 기준 + +비용 영역은 사용량 모니터링 이후에 세부 기준을 설정한다. + +| 평가 영역 | 설명 | 우선순위 | 상(3) | 중(2) | 하(1) | +| ---------- | ------------------------------------- | ---- | ------------------------------------- | ------------------------------------------------------- | ------------------------------------------------ | +| 사실 정확성 | 문서에 근거한 사실을 정확하게 답변하는가? | 1 | 문서에 근거하여 사실을 정확하게 답변한다. | 문서에 근거가 있으나 내용이 왜곡되거나 부정확하다. | 문서에 근거하지 않은 사실을 답변하거나 잘못된 정보를 제공한다. | +| 보안성 | 권한이 없거나 노출되면 안 되는 민감/기밀 정보를 드러내지 않는가? | 2 | 민감/기밀 정보를 드러내지 않고 허용된 정보 범위 내에서 답변한다. | 민감도는 낮지만 내부 문서명, 내부 정책 표현, 상담 로그 일부 등 불필요한 내부 정보를 드러낸다. | 개인정보, 기밀 문서, 보안과 직결된 내부 정보 등 노출되면 안 되는 정보를 드러낸다. | +| 맥락 적합성 | 사용자 질문의 의도와 대화 맥락에 맞게 답변하는가? | 3 | 사용자의 질문 의도와 대화 맥락에 맞는 정보를 답변한다. | 질문과 일부 관련은 있으나 사용자의 의도나 맥락을 충분히 반영하지 못한다. | 사용자의 질문 해결에 필요한 정보가 아닌 다른 정보를 답변한다. | +| 비용(토큰 사용량) | 챗봇 호출에 필요한 토큰 사용량이 과도하지 않은가? | 4 | - | - | - | +| 응답 속도 | 사용자가 불편을 느끼지 않을 정도로 신속하게 응답하는가? | 5 | 5초 이내에 답변한다. | 5초 초과, 10초 이내에 답변한다. | 10초를 초과하여 답변한다. | +| 답변 명확성 | 답변이 명확하고 이해하기 쉬우며 혼란을 주지 않는가? | 6 | 표준 어투로 이해하기 쉽고 일관되며 명확한 답변을 제공한다. | 일부 표현이 모호하거나 일관되지 않아 이해에 약간의 어려움이 있다. | 과도한 전문 용어, 모호한 표현, 불명확한 구조로 인해 이해하거나 활용하기 어렵다. | diff --git a/README.md b/README.md index 17b9d19..368e19b 100644 --- a/README.md +++ b/README.md @@ -1,92 +1,21 @@ -# Stage 1: 고객지원 챗봇 만들기 +# 프로젝트 개요 -Spring AI로 FAQ 챗봇을 직접 만드는 실습입니다. +초록코퍼레이션 고객 응대 챗봇입니다. -커리큘럼 소개는 [SYLLABUS.md](SYLLABUS.md)를, 진행 방법은 [GUIDE.md](GUIDE.md)를 참고하세요. +# 애플리케이션 실행 ---- +## 실행 옵션 -## 필요한 것 - -| 항목 | 비고 | -|------|------| -| Java 17+ | `java -version`으로 확인 | -| OpenAI API 키 | [platform.openai.com](https://platform.openai.com)에서 발급 | -| IDE | IntelliJ IDEA 권장 (VS Code + Java Extension Pack도 가능) | - -- Spring Boot로 REST API를 만들어 본 경험이 있으면 됩니다 -- AI/ML 사전 지식은 없어도 됩니다 -- 끝까지 해도 API 비용은 **$1-5** 안쪽입니다 (GPT-4.1-nano) - ---- - -## 빠른 시작 - -```bash -# 프로젝트 루트에서 -cp .env.example .env # OpenAI API 키 입력 -./gradlew bootRun -``` - -서버가 `http://localhost:8080`에서 시작됩니다. - -그 다음 [mission/MISSION.md](mission/MISSION.md)를 여세요. 거기서부터 시작입니다. - ---- - -## 평가 - -`data/test_questions.json`에 100개의 테스트 질문이 있습니다 (easy 30 / medium 44 / hard 26). - -서버를 띄운 상태에서 평가 스크립트를 실행할 수 있습니다: +애플리케이션 실행 시 VectorStore 임베딩을 수행하려면 인자로 --mode=embedding 옵션을 주어야합니다. +임베딩 수행하지 않을 경우 옵션을 제외합니다. +## 실행 명령어 ```bash -cd data - -# Python 환경 준비 -python -m venv .venv -.venv/bin/pip install openai qdrant-client python-dotenv +# bootrun 실행 시 +./gradlew bootrun +./gradlew bootrun --args='--mode=embedding' -# 평가 실행 (judge 모델 gpt-4o-mini 사용, 100문항 기준 약 $0.5~1 추가 비용) -.venv/bin/python evaluate_rag.py +# jar 실행 시 +java -jar +java -jar --mode=embedding ``` - ---- - -## 프로젝트 구조 - -``` -spring-ai-bootcamp-basic/ -├── mission/ -│ ├── MISSION.md # 미션 설명 (여기서 시작) -│ └── wall-report.md # 벽 리포트 (마지막에 작성) -├── hints/ -│ ├── HINT_01.md ~ HINT_06.md # 막혔을 때 열어보세요 -├── data/ -│ ├── layer1_faq/ # 공식 FAQ 문서 -│ ├── layer2_policies/ # 사내 정책 문서 -│ ├── layer3_chatlogs/ # 고객 상담 로그 -│ └── test_questions.json # 평가용 질문 100개 -├── src/ -│ └── main/java/com/cholog/bootcamp/ -│ └── Application.java # 여기서부터 만드세요 -├── SYLLABUS.md # 커리큘럼 소개 ← 과정 전체 그림 -├── GUIDE.md # 진행 가이드 ← 미션 중 참고 -└── build.gradle -``` - ---- - -## 자주 묻는 질문 - -**Q: API 키 없이 시작할 수 있나요?** - -코드 작성과 컴파일은 가능하지만, 실제 실행에는 OpenAI API 키가 필요합니다. - -**Q: FAQ 데이터가 영어인데 질문은 한국어로 해도 되나요?** - -네. GPT-4.1-nano는 교차 언어 이해가 가능합니다. - -**Q: 어떤 도구를 써도 되나요?** - -네. 구현 방법에 제약은 없습니다. diff --git a/SYLLABUS.md b/SYLLABUS.md index 61d3021..1bf155f 100644 --- a/SYLLABUS.md +++ b/SYLLABUS.md @@ -59,7 +59,7 @@ Easy는 그럭저럭. Medium부터 급락합니다. - 챗봇 구현·실행: **$1~5** 안쪽 (GPT-4.1-nano 기준) - 평가 스크립트까지 돌리면: **+$0.5~1** (judge 모델 `gpt-4o-mini`) -API 키 발급과 환경 설정은 [README.md](README.md#필요한-것)를 참고하세요. +API 키 발급과 환경 설정은 [BASE-README.md](BASE-README.md#필요한-것)를 참고하세요. --- diff --git a/build.gradle b/build.gradle index 941e596..7309f0d 100644 --- a/build.gradle +++ b/build.gradle @@ -25,7 +25,13 @@ dependencyManagement { dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' + implementation 'org.springframework.boot:spring-boot-starter-thymeleaf' implementation 'org.springframework.ai:spring-ai-starter-model-openai' + implementation 'org.springframework.ai:spring-ai-advisors-vector-store' + implementation 'org.springframework.ai:spring-ai-markdown-document-reader' + implementation 'org.springframework.ai:spring-ai-rag' + compileOnly 'org.projectlombok:lombok' + annotationProcessor 'org.projectlombok:lombok' testImplementation 'org.springframework.boot:spring-boot-starter-test' } diff --git a/data/evaluate.py b/data/evaluate.py index ed941cd..c58325d 100644 --- a/data/evaluate.py +++ b/data/evaluate.py @@ -170,7 +170,7 @@ def main(): if args.verbose: print(f"[{qid}] {marker} ({tier}) {question_ko[:40]}...") if score == 0: - print(f" 이유: {judgment.get('reason', '')[:80]}") + print(f" 이유: {judgment.get('reason', '')[:160]}") # 진행률 (10개마다) if not args.verbose and (i + 1) % 10 == 0: diff --git a/data/evaluate2.py b/data/evaluate2.py new file mode 100644 index 0000000..233f710 --- /dev/null +++ b/data/evaluate2.py @@ -0,0 +1,292 @@ +""" +챗봇 품질 평가 스크립트 + +실행 중인 서버(localhost:8080)에 테스트 질문을 보내고, +LLM 판정으로 정확도를 측정합니다. + +사전 준비: + python -m venv .venv + .venv/bin/pip install openai python-dotenv requests + +실행: + # 서버가 localhost:8080에서 실행 중이어야 합니다 + .venv/bin/python evaluate2.py + .venv/bin/python evaluate2.py --verbose # 질문별 상세 출력 + .venv/bin/python evaluate2.py --limit 10 # 처음 10개만 평가 + +비용: + judge 모델(gpt-4o-mini) 사용, 100문항 기준 약 $0.3~0.5 +""" + +import json +import os +import argparse +import time +from pathlib import Path + +import requests +from dotenv import dotenv_values +from openai import OpenAI + +# ─── 설정 ───────────────────────────────────────────────────────────────────── + +DATA_DIR = Path(__file__).parent +ROOT_DIR = DATA_DIR.parent + +SERVER_URL = "http://localhost:8080/api/chat" +JUDGE_MODEL = "gpt-4o-mini" + +# .env에서 API 키 로드 +env_path = ROOT_DIR / ".env" +env_vars = dotenv_values(env_path) +OPENAI_API_KEY = env_vars.get("OPENAI_API_KEY") or os.environ.get("OPENAI_API_KEY") + +openai_client = OpenAI(api_key=OPENAI_API_KEY) + + +# ─── 서버 호출 ──────────────────────────────────────────────────────────────── + +def ask_server(question: str) -> dict | None: + """학습자의 챗봇 서버에 질문을 보냅니다.""" + try: + resp = requests.post( + SERVER_URL, + json={"question": question}, + timeout=60, + ) + if resp.status_code == 200: + return resp.json() + else: + print(f" [ERROR] HTTP {resp.status_code}: {resp.text[:100]}") + return None + except requests.exceptions.ConnectionError: + print(f" [ERROR] 서버에 연결할 수 없습니다: {SERVER_URL}") + return None + except requests.exceptions.Timeout: + print(f" [ERROR] 타임아웃 (60초)") + return None + + +# ─── LLM 판정 ───────────────────────────────────────────────────────────────── + +def judge_answer(question: str, expected: str, actual: str) -> dict: + """LLM으로 답변의 사실적 일치도를 판정합니다.""" + prompt = f"""당신은 FAQ 챗봇 답변의 품질을 평가하는 판정자입니다. + +질문: {question} + +기대 답변 (정답): {expected} + +실제 답변 (챗봇): {actual} + +실제 답변이 기대 답변과 사실적으로 일치하는지 평가하세요. +- 표현이 달라도 핵심 사실이 같으면 정답입니다 +- 핵심 사실이 빠져있거나 틀렸으면 오답입니다 +- 부분적으로만 맞으면 오답으로 처리하세요 + +JSON으로만 응답하세요: +{{"score": 1, "reason": "..."}} (정답) +{{"score": 0, "reason": "..."}} (오답) +""" + + resp = openai_client.chat.completions.create( + model=JUDGE_MODEL, + messages=[{"role": "user", "content": prompt}], + temperature=0, + response_format={"type": "json_object"}, + ) + + try: + return json.loads(resp.choices[0].message.content) + except json.JSONDecodeError: + return {"score": 0, "reason": "판정 파싱 실패"} + + +# ─── 메인 ───────────────────────────────────────────────────────────────────── + +# questions_filename = "test_questions.json" +questions_filename = "manual_review_questions.json" + +def main(): + parser = argparse.ArgumentParser(description="챗봇 품질 평가") + parser.add_argument("--verbose", action="store_true", help="질문별 상세 출력") + parser.add_argument("--limit", type=int, default=0, help="평가할 질문 수 제한 (0=전체)") + args = parser.parse_args() + + # 테스트 질문 로드 + questions_path = DATA_DIR / questions_filename + with open(questions_path) as f: + questions = json.load(f) + + if args.limit > 0: + questions = questions[:args.limit] + + print(f"=== 챗봇 품질 평가 시작 ===") + print(f"서버: {SERVER_URL}") + print(f"질문 수: {len(questions)}") + print() + + # 서버 연결 확인 + test_resp = ask_server("test") + if test_resp is None: + print("서버에 연결할 수 없습니다. 서버가 실행 중인지 확인하세요:") + print(f" ./gradlew bootRun") + return + + elapsed_times = {} + actual_answers = {} + error_ids = [] + run_start_time = time.time() + + for i, q in enumerate(questions): + qid = q.get("id", f"Q{i+1}") # 질문 id + question_ko = q["question_ko"] # 질문 + + # 서버에 질문 + request_start_time = time.time() + response = ask_server(question_ko) + if response is None: + error_ids.append(qid) + elapsed_times[qid] = None + actual_answers[qid] = "" + if args.verbose: + print(f"[{qid}] ERROR — 서버 응답 없음") + continue + + elapsed = time.time() - request_start_time # 답변 소요 시간 + elapsed_times[qid] = elapsed + + actual_answer = response.get("answer", "") # 실제 답변 + actual_answers[qid] = actual_answer + + # 결과 저장 + total_elapsed = time.time() - run_start_time + total = len(questions) + answered = total - len(error_ids) + answered_elapsed_times = [ + elapsed_time + for elapsed_time in elapsed_times.values() + if elapsed_time is not None + ] + + by_tier = {} + by_wall_type = {} + review_items = [] + + for i, q in enumerate(questions): + qid = q.get("id", f"Q{i+1}") + tier = q.get("tier", "unknown") + wall_type = q.get("wall_type") or "null" + elapsed_time = elapsed_times.get(qid) + + by_tier[tier] = by_tier.get(tier, 0) + 1 + by_wall_type[wall_type] = by_wall_type.get(wall_type, 0) + 1 + + if elapsed_time is None: + response_speed_score = None + elif elapsed_time <= 5: + response_speed_score = 3 + elif elapsed_time <= 10: + response_speed_score = 2 + else: + response_speed_score = 1 + + review_items.append({ + "id": qid, + "tier": tier, + "wall_type": q.get("wall_type"), + "question_ko": q.get("question_ko", ""), + "question_en": q.get("question_en", ""), + "expected_answer": q.get("expected_answer", ""), + "actual_answer": actual_answers.get(qid, ""), + "elapsed_seconds": elapsed_time, + "request_error": qid in error_ids, + "auto_evaluation": { + "response_speed": { + "score": response_speed_score, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과", + }, + }, + "manual_evaluation": { + "fact_accuracy": {"score": None, "comment": ""}, + "security": {"score": None, "comment": ""}, + "context_fit": {"score": None, "comment": ""}, + "clarity": {"score": None, "comment": ""}, + }, + "manual_summary": { + "overall_score": None, + "comment": "", + }, + }) + + result_file = DATA_DIR / "manual_review_result.json" + with open(result_file, "w") as f: + json.dump({ + "source_questions": questions_filename, + "total": total, + "answered": answered, + "error": len(error_ids), + "error_ids": error_ids, + "elapsed_seconds": total_elapsed, + "average_response_seconds": ( + sum(answered_elapsed_times) / max(len(answered_elapsed_times), 1) + ), + "excluded_kpi": ["cost"], + "scoring_scale": { + "3": "상", + "2": "중", + "1": "하", + }, + "kpi_criteria": [ + { + "key": "fact_accuracy", + "name": "사실 정확성", + "priority": 1, + "high_3": "문서에 근거하여 사실을 정확하게 답변한다.", + "middle_2": "문서에 근거가 있으나 내용이 왜곡되거나 부정확하다.", + "low_1": "문서에 근거하지 않은 사실을 답변하거나 잘못된 정보를 제공한다.", + }, + { + "key": "security", + "name": "보안성", + "priority": 2, + "high_3": "민감/기밀 정보를 드러내지 않고 허용된 정보 범위 내에서 답변한다.", + "middle_2": "민감도는 낮지만 내부 문서명, 내부 정책 표현, 상담 로그 일부 등 불필요한 내부 정보를 드러낸다.", + "low_1": "개인정보, 기밀 문서, 보안과 직결된 내부 정보 등 노출되면 안 되는 정보를 드러낸다.", + }, + { + "key": "context_fit", + "name": "맥락 적합성", + "priority": 3, + "high_3": "사용자의 질문 의도와 대화 맥락에 맞는 정보를 답변한다.", + "middle_2": "질문과 일부 관련은 있으나 사용자의 의도나 맥락을 충분히 반영하지 못한다.", + "low_1": "사용자의 질문 해결에 필요한 정보가 아닌 다른 정보를 답변한다.", + }, + { + "key": "response_speed", + "name": "응답 속도", + "priority": 5, + "high_3": "5초 이내에 답변한다.", + "middle_2": "5초 초과, 10초 이내에 답변한다.", + "low_1": "10초를 초과하여 답변한다.", + }, + { + "key": "clarity", + "name": "답변 명확성", + "priority": 6, + "high_3": "표준 어투로 이해하기 쉽고 일관되며 명확한 답변을 제공한다.", + "middle_2": "일부 표현이 모호하거나 일관되지 않아 이해에 약간의 어려움이 있다.", + "low_1": "과도한 전문 용어, 모호한 표현, 불명확한 구조로 인해 이해하거나 활용하기 어렵다.", + }, + ], + "counts": { + "by_tier": by_tier, + "by_wall_type": by_wall_type, + }, + "items": review_items, + }, f, indent=2, ensure_ascii=False) + print(f"\n결과 저장: {result_file}") + + +if __name__ == "__main__": + main() diff --git a/data/manual_review_questions.json b/data/manual_review_questions.json new file mode 100644 index 0000000..eeacb99 --- /dev/null +++ b/data/manual_review_questions.json @@ -0,0 +1,252 @@ +[ + { + "id": "Q001", + "question_ko": "주문 취소는 어떻게 하나요?", + "question_en": "How do I cancel my order?", + "expected_answer": "Go to My Orders and tap Cancel Order while the status is Preparing. Cancellation is instant with a full refund. Once the order is Shipped, cancellation is not possible and you must initiate a return after delivery.", + "tier": "easy", + "source_layers": [ + "faq" + ], + "primary_intent": "cancel_order", + "wall_type": null + }, + { + "id": "Q007", + "question_ko": "냉장 배송 요일이 따로 있나요?", + "question_en": "Are there restricted days for cold-chain delivery?", + "expected_answer": "Yes. Cold-chain (fresh and frozen) delivery is available Monday through Thursday only, to prevent packages from sitting in transit over a weekend. A flat cold-chain fee of 4,000 won applies regardless of order amount or membership tier.", + "tier": "easy", + "source_layers": [ + "faq" + ], + "primary_intent": "delivery_options", + "wall_type": null + }, + { + "id": "Q016", + "question_ko": "마켓플레이스 상품도 14일 안에 반품할 수 있나요?", + "question_en": "Can I return a Marketplace item within 14 days?", + "expected_answer": "Marketplace items follow the individual seller's return policy, not Cholog's standard 14-day window. Return windows vary from 3 to 30 days depending on the seller. Always check the seller's policy on the product page before purchasing.", + "tier": "medium", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "check_refund_policy", + "wall_type": "contradiction" + }, + { + "id": "Q017", + "question_ko": "VIP인데 냉장 상품 배송비도 무료인가요?", + "question_en": "As a VIP member, is cold-chain shipping also free?", + "expected_answer": "No. VIP members receive free Priority shipping on standard orders, but cold-chain delivery carries a flat 4,000 won fee for all members regardless of tier. The cold-chain fee applies because it uses a separate courier (CJ Logistics Fresh) with specialized packaging.", + "tier": "medium", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "delivery_options", + "wall_type": "conditional" + }, + { + "id": "Q018", + "question_ko": "포인트 적립률이 몇 퍼센트인가요?", + "question_en": "What is the loyalty point earning rate?", + "expected_answer": "The current earning rates are: Standard tier 1%, Plus tier 3%, VIP tier 5%. These rates were updated in June 2023; the previous rates were higher (3%/5%/7%) but are no longer in effect.", + "tier": "medium", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "loyalty_points", + "wall_type": "versioning" + }, + { + "id": "Q022", + "question_ko": "반품 금액이 크면 등급이 떨어질 수 있나요?", + "question_en": "Can returning items cause a tier downgrade?", + "expected_answer": "Yes. Returned order amounts are deducted from your annual spending total. If the deduction drops your total below the tier threshold (Plus: 200,000 won, VIP: 800,000 won), your tier is downgraded at the next monthly recalculation on the 1st of the following month.", + "tier": "medium", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "membership_tiers", + "wall_type": "conditional" + }, + { + "id": "Q024", + "question_ko": "플러스 등급 기준이 연 15만원인가요, 20만원인가요?", + "question_en": "Is the Plus tier threshold 150,000 won or 200,000 won per year?", + "expected_answer": "The current Plus tier threshold is 200,000 won per year (as of January 2024). The threshold was previously 150,000 won in 2023. The current VIP threshold is 800,000 won (previously 600,000 won).", + "tier": "medium", + "source_layers": [ + "policy" + ], + "primary_intent": "membership_tiers", + "wall_type": "versioning" + }, + { + "id": "Q032", + "question_ko": "구독 중이면 스탠다드인데도 2만원 미만 주문에 배송비 무료인가요?", + "question_en": "If I have a subscription but I'm Standard tier, is shipping still free on orders under 20,000 won?", + "expected_answer": "Yes. Active subscribers get free shipping on all orders regardless of order amount or membership tier. The 20,000 won free shipping threshold applies only to non-subscriber Standard members.", + "tier": "medium", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "subscription", + "wall_type": "conditional" + }, + { + "id": "Q034", + "question_ko": "걍 환불해주세요 ㅠㅠ", + "question_en": "I just want a refund please", + "expected_answer": "To process a refund, please go to My Orders, select the relevant order, and tap Request Return. Choose your reason, upload at least one photo, and select a pickup date. For change of mind, a 3,000 won handling fee is deducted. Kakao Pay refunds are typically instant after inspection.", + "tier": "hard", + "source_layers": [ + "faq" + ], + "primary_intent": "get_refund", + "wall_type": "cross_language" + }, + { + "id": "Q036", + "question_ko": "배송 지연됐는데 보상받은 사례가 있나요?", + "question_en": "Have other customers received compensation for delivery delays?", + "expected_answer": "Yes. Based on customer support records, delivery delays exceeding 3 business days (non-island) typically result in a 2,000 won coupon compensation. For significant delays or damaged items, replacements and larger coupons have been issued.", + "tier": "hard", + "source_layers": [ + "chatlog", + "policy" + ], + "primary_intent": "delivery_period", + "wall_type": "precedent" + }, + { + "id": "Q037", + "question_ko": "저번에 물어본 배송 건 어떻게 됐어요?", + "question_en": "What happened to the shipping issue I asked about last time?", + "expected_answer": "I don't have access to your previous conversation history. Please provide your order number and I can look up the current status of your shipment directly.", + "tier": "hard", + "source_layers": [ + "chatlog" + ], + "primary_intent": "track_order", + "wall_type": "memory" + }, + { + "id": "Q038", + "question_ko": "냉장 상품 오늘 금요일에 주문하면 언제 와요?", + "question_en": "If I order a cold-chain product today (Friday), when will it arrive?", + "expected_answer": "Cold-chain delivery is only available Monday through Thursday. An order placed on Friday will be dispatched the following Monday. Add 1-2 business days for delivery, so the earliest expected arrival would be Tuesday or Wednesday of next week.", + "tier": "hard", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "delivery_period", + "wall_type": "conditional" + }, + { + "id": "Q045", + "question_ko": "VIP 30일 반품 가능하다고 들었는데요", + "question_en": "I heard VIP members can return items within 30 days — is that true?", + "expected_answer": "The standard return window for all members including VIP is 14 calendar days. However, in certain exceptional cases (documented absence during return window, defect discovered after extended use), VIP members may be granted an extended return window at CS agent discretion. This is not a publicized policy and is applied case-by-case.", + "tier": "hard", + "source_layers": [ + "faq", + "policy" + ], + "primary_intent": "check_refund_policy", + "wall_type": "contradiction" + }, + { + "id": "Q050", + "question_ko": "반품 신청 후 환불받기까지 가장 빠른 경우 얼마나 걸려요?", + "question_en": "What is the fastest possible refund timeline after initiating a return?", + "expected_answer": "The fastest scenario: courier picks up the item, inspection completes in 1 business day, and Kakao Pay or Naver Pay refund is processed instantly. Minimum total time is approximately 2-3 business days from return request to refund. Credit card refunds take an additional 3-5 business days after inspection.", + "tier": "medium", + "source_layers": [ + "faq" + ], + "primary_intent": "get_refund", + "wall_type": null + }, + { + "id": "Q058", + "question_ko": "어떤 결제 수단을 지원하나요?", + "question_en": "What payment methods are supported?", + "expected_answer": "Supported payment methods include credit/debit cards, Kakao Pay, Naver Pay, real-time bank transfer, loyalty points, and discount coupons. Points and coupons can be combined with other payment methods. Virtual account (가상계좌) is not supported.", + "tier": "easy", + "source_layers": [ + "faq" + ], + "primary_intent": "check_payment_methods", + "wall_type": null + }, + { + "id": "Q085", + "question_ko": "준비중 상태에서 취소는 즉시 처리되나요?", + "question_en": "Is cancellation immediate when the order is in Preparing status?", + "expected_answer": "Yes. Canceling an order while it is in Preparing status is processed instantly with no fee. Some agents have mistakenly advised that cancellation takes 1-2 business days — this is incorrect. You can cancel directly in the app and the status updates immediately.", + "tier": "medium", + "source_layers": [ + "chatlog", + "policy" + ], + "primary_intent": "cancel_order", + "wall_type": "noise" + }, + { + "id": "Q088", + "question_ko": "VIP 포인트가 7%라고 어디선가 봤는데 맞나요?", + "question_en": "I saw somewhere that VIP members earn 7% loyalty points — is that accurate?", + "expected_answer": "No. The current VIP earning rate is 5%. The previous rate was 7% under the old policy (Standard 3%, Plus 5%, VIP 7%), but this was updated in early 2024. Current rates are Standard 1%, Plus 3%, VIP 5%. Some older documents or chat logs may still reference the outdated 7% figure.", + "tier": "hard", + "source_layers": [ + "policy", + "chatlog" + ], + "primary_intent": "loyalty_points", + "wall_type": "noise" + }, + { + "id": "Q097", + "question_ko": "반품 기간이 10일이라고 들었는데 맞나요?", + "question_en": "I heard the return window is 10 days — is that right?", + "expected_answer": "No. The current return window is 14 calendar days (policy v3, effective April 2024). A 10-day window existed under policy v2 and a 7-day window under policy v1, but neither is in effect. If you encountered this information in an older document or a chat transcript, it is outdated.", + "tier": "hard", + "source_layers": [ + "policy" + ], + "primary_intent": "check_refund_policy", + "wall_type": "versioning" + }, + { + "id": "Q116", + "question_ko": "환불어케함? 반품하고싶은데", + "question_en": "How do I return something?", + "expected_answer": "Go to My Orders in the app or website, tap the order, select Request Return, choose a reason, and upload a photo. Courier pickup is free for defect returns; a 3,000 won fee applies for non-defect returns.", + "tier": "medium", + "source_layers": [ + "faq" + ], + "primary_intent": "get_refund", + "wall_type": "cross_language" + }, + { + "id": "Q150", + "question_ko": "불량품받았는데ㅠ 어케함 환불됨?", + "question_en": "I received a defective product what should I do?", + "expected_answer": "Request a return through My Orders and select Defective/Wrong Item as the reason. Upload photos of the defect. Courier pickup is free for defect returns and a full refund will be issued.", + "tier": "medium", + "source_layers": [ + "faq" + ], + "primary_intent": "wrong_item_received", + "wall_type": "cross_language" + } +] diff --git a/data/manual_review_result.json b/data/manual_review_result.json new file mode 100644 index 0000000..abc551a --- /dev/null +++ b/data/manual_review_result.json @@ -0,0 +1,858 @@ +{ + "source_questions": "manual_review_questions.json", + "total": 20, + "answered": 20, + "error": 0, + "error_ids": [], + "elapsed_seconds": 28.40176486968994, + "average_response_seconds": 1.4200799942016602, + "excluded_kpi": [ + "cost" + ], + "scoring_scale": { + "3": "상", + "2": "중", + "1": "하" + }, + "kpi_criteria": [ + { + "key": "fact_accuracy", + "name": "사실 정확성", + "priority": 1, + "high_3": "문서에 근거하여 사실을 정확하게 답변한다.", + "middle_2": "문서에 근거가 있으나 내용이 왜곡되거나 부정확하다.", + "low_1": "문서에 근거하지 않은 사실을 답변하거나 잘못된 정보를 제공한다." + }, + { + "key": "security", + "name": "보안성", + "priority": 2, + "high_3": "민감/기밀 정보를 드러내지 않고 허용된 정보 범위 내에서 답변한다.", + "middle_2": "민감도는 낮지만 내부 문서명, 내부 정책 표현, 상담 로그 일부 등 불필요한 내부 정보를 드러낸다.", + "low_1": "개인정보, 기밀 문서, 보안과 직결된 내부 정보 등 노출되면 안 되는 정보를 드러낸다." + }, + { + "key": "context_fit", + "name": "맥락 적합성", + "priority": 3, + "high_3": "사용자의 질문 의도와 대화 맥락에 맞는 정보를 답변한다.", + "middle_2": "질문과 일부 관련은 있으나 사용자의 의도나 맥락을 충분히 반영하지 못한다.", + "low_1": "사용자의 질문 해결에 필요한 정보가 아닌 다른 정보를 답변한다." + }, + { + "key": "response_speed", + "name": "응답 속도", + "priority": 5, + "high_3": "5초 이내에 답변한다.", + "middle_2": "5초 초과, 10초 이내에 답변한다.", + "low_1": "10초를 초과하여 답변한다." + }, + { + "key": "clarity", + "name": "답변 명확성", + "priority": 6, + "high_3": "표준 어투로 이해하기 쉽고 일관되며 명확한 답변을 제공한다.", + "middle_2": "일부 표현이 모호하거나 일관되지 않아 이해에 약간의 어려움이 있다.", + "low_1": "과도한 전문 용어, 모호한 표현, 불명확한 구조로 인해 이해하거나 활용하기 어렵다." + } + ], + "counts": { + "by_tier": { + "easy": 3, + "medium": 10, + "hard": 7 + }, + "by_wall_type": { + "null": 4, + "contradiction": 2, + "conditional": 4, + "versioning": 3, + "cross_language": 3, + "precedent": 1, + "memory": 1, + "noise": 2 + } + }, + "items": [ + { + "id": "Q001", + "tier": "easy", + "wall_type": null, + "question_ko": "주문 취소는 어떻게 하나요?", + "question_en": "How do I cancel my order?", + "expected_answer": "Go to My Orders and tap Cancel Order while the status is Preparing. Cancellation is instant with a full refund. Once the order is Shipped, cancellation is not possible and you must initiate a return after delivery.", + "actual_answer": "주문 취소는 \"My Orders\"에서 해당 주문의 상태가 \"Preparing\"일 때 \"Cancel Order\"를 탭하시면 즉시 취소됩니다. 취소가 완료되면 결제하신 금액이 원래 결제수단으로 환불됩니다. 단, 주문 상태가 \"Shipped\"로 변경된 경우에는 취소가 불가능하며, 배송이 완료된 후에는 반품 절차를 진행하셔야 합니다.", + "elapsed_seconds": 2.1312150955200195, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "한국어로 물어봤으나 영문 섹션 안내" + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q007", + "tier": "easy", + "wall_type": null, + "question_ko": "냉장 배송 요일이 따로 있나요?", + "question_en": "Are there restricted days for cold-chain delivery?", + "expected_answer": "Yes. Cold-chain (fresh and frozen) delivery is available Monday through Thursday only, to prevent packages from sitting in transit over a weekend. A flat cold-chain fee of 4,000 won applies regardless of order amount or membership tier.", + "actual_answer": "냉장 배송은 월요일부터 목요일까지만 가능합니다. 금요일, 토요일, 일요일 및 공휴일에는 냉장 배송이 제공되지 않습니다.", + "elapsed_seconds": 0.8133330345153809, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 3, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 15, + "comment": "" + } + }, + { + "id": "Q016", + "tier": "medium", + "wall_type": "contradiction", + "question_ko": "마켓플레이스 상품도 14일 안에 반품할 수 있나요?", + "question_en": "Can I return a Marketplace item within 14 days?", + "expected_answer": "Marketplace items follow the individual seller's return policy, not Cholog's standard 14-day window. Return windows vary from 3 to 30 days depending on the seller. Always check the seller's policy on the product page before purchasing.", + "actual_answer": "네, 마켓플레이스 상품도 14일 이내에 반품하실 수 있습니다. 다만, 마켓플레이스 상품은 판매자의 개별 반품 정책에 따르기 때문에, 판매자 페이지에서 구체적인 반품 기간과 조건을 확인하셔야 합니다.", + "elapsed_seconds": 0.8151259422302246, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 2, + "comment": "처음에 잘못된 답변함" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + }, + { + "id": "Q017", + "tier": "medium", + "wall_type": "conditional", + "question_ko": "VIP인데 냉장 상품 배송비도 무료인가요?", + "question_en": "As a VIP member, is cold-chain shipping also free?", + "expected_answer": "No. VIP members receive free Priority shipping on standard orders, but cold-chain delivery carries a flat 4,000 won fee for all members regardless of tier. The cold-chain fee applies because it uses a separate courier (CJ Logistics Fresh) with specialized packaging.", + "actual_answer": "VIP 고객님께서는 냉장 상품 배송비도 무료가 아닙니다. 냉장 상품 배송에는 4,000원의 냉장 배송비가 부과되며, 이는 회원 등급과 관계없이 적용됩니다.", + "elapsed_seconds": 0.934063196182251, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "`VIP 고객님께서는 냉장 상품 배송비도` 라는 표현이 다른 상품 배송비도 무료가 아니라는 표현 같아서 애매하다" + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q018", + "tier": "medium", + "wall_type": "versioning", + "question_ko": "포인트 적립률이 몇 퍼센트인가요?", + "question_en": "What is the loyalty point earning rate?", + "expected_answer": "The current earning rates are: Standard tier 1%, Plus tier 3%, VIP tier 5%. These rates were updated in June 2023; the previous rates were higher (3%/5%/7%) but are no longer in effect.", + "actual_answer": "포인트 적립률은 회원 등급에 따라 다릅니다. \n- Standard 등급: 1% \n- Plus 등급: 3% \n- VIP 등급: 5% \n\n즉, 주문 금액의 1%, 3%, 또는 5%에 해당하는 포인트가 적립됩니다.", + "elapsed_seconds": 1.0017859935760498, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "`즉, 주문 금액의 1%, 3%, 또는 5%에 해당하는 포인트가 적립됩니다.`를 마지막에 다시 말하는건 굳이 싶음." + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q022", + "tier": "medium", + "wall_type": "conditional", + "question_ko": "반품 금액이 크면 등급이 떨어질 수 있나요?", + "question_en": "Can returning items cause a tier downgrade?", + "expected_answer": "Yes. Returned order amounts are deducted from your annual spending total. If the deduction drops your total below the tier threshold (Plus: 200,000 won, VIP: 800,000 won), your tier is downgraded at the next monthly recalculation on the 1st of the following month.", + "actual_answer": "반품 금액이 크다고 해서 등급이 떨어지지는 않습니다. 등급은 연간 총 구매 금액(지출액)을 기준으로 평가되며, 반품으로 인해 차감된 금액이 반영됩니다. 그러나 반품 금액이 크더라도, 단순히 반품 금액만으로 등급이 바로 떨어지지는 않으며, 연간 구매 총액이 기준에 미달할 경우에만 등급이 조정됩니다.", + "elapsed_seconds": 1.4721360206604004, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 2, + "comment": "반품 금액이 크면 등급 떨어질 수 있음" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "첫 결론과 뒤 부가 설명이 일치하지 않아 애매" + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + }, + { + "id": "Q024", + "tier": "medium", + "wall_type": "versioning", + "question_ko": "플러스 등급 기준이 연 15만원인가요, 20만원인가요?", + "question_en": "Is the Plus tier threshold 150,000 won or 200,000 won per year?", + "expected_answer": "The current Plus tier threshold is 200,000 won per year (as of January 2024). The threshold was previously 150,000 won in 2023. The current VIP threshold is 800,000 won (previously 600,000 won).", + "actual_answer": "플러스 등급의 연간 소비 기준은 20만원 이상입니다.", + "elapsed_seconds": 1.0679090023040771, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 3, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 15, + "comment": "" + } + }, + { + "id": "Q032", + "tier": "medium", + "wall_type": "conditional", + "question_ko": "구독 중이면 스탠다드인데도 2만원 미만 주문에 배송비 무료인가요?", + "question_en": "If I have a subscription but I'm Standard tier, is shipping still free on orders under 20,000 won?", + "expected_answer": "Yes. Active subscribers get free shipping on all orders regardless of order amount or membership tier. The 20,000 won free shipping threshold applies only to non-subscriber Standard members.", + "actual_answer": "구독 중이시더라도 스탠다드(일반) 배송은 2만원 이상 주문 시 무료입니다. 2만원 미만 주문일 경우에는 배송비 2,500원이 부과됩니다. 따라서, 구독 상태와 상관없이 주문 금액이 20,000원 이상이면 무료 배송이 적용됩니다.", + "elapsed_seconds": 1.4352450370788574, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 2, + "comment": "등급을 무시한 2만원 미만 배송비 안내" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 1, + "comment": "뭔 말인지 모르겠음. 질문은 미만을 물어봤는데 이상을 답함." + } + }, + "manual_summary": { + "overall_score": 12, + "comment": "" + } + }, + { + "id": "Q034", + "tier": "hard", + "wall_type": "cross_language", + "question_ko": "걍 환불해주세요 ㅠㅠ", + "question_en": "I just want a refund please", + "expected_answer": "To process a refund, please go to My Orders, select the relevant order, and tap Request Return. Choose your reason, upload at least one photo, and select a pickup date. For change of mind, a 3,000 won handling fee is deducted. Kakao Pay refunds are typically instant after inspection.", + "actual_answer": "고객님, 환불 요청에 대해 안내드리겠습니다. \n반품 정책에 따라, 제품이 개봉되지 않았거나, 개봉하셨더라도 사용하지 않으셨으며 원래 포장과 모든 액세서리, 문서가 포함되어 있어야 합니다. \n\n환불 진행을 위해서는 먼저 반품 신청이 필요하며, 반품 사유에 따라 처리 방법이 달라집니다. \n단, 일부 상품은 반품이 불가하거나, 반품 시 수수료가 발생할 수 있습니다. \n\n구체적인 상품 정보와 반품 사유를 알려주시면, 보다 정확한 안내를 도와드리겠습니다. \n혹시 반품이 어려운 상품이거나, 이미 반품 절차를 진행하셨다면 말씀해 주세요.", + "elapsed_seconds": 1.8564298152923584, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 2, + "comment": "환불 방법을 안내해야하지 않음" + }, + "clarity": { + "score": 2, + "comment": "주저리 주저리. 정제되지 않은 느낌." + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + }, + { + "id": "Q036", + "tier": "hard", + "wall_type": "precedent", + "question_ko": "배송 지연됐는데 보상받은 사례가 있나요?", + "question_en": "Have other customers received compensation for delivery delays?", + "expected_answer": "Yes. Based on customer support records, delivery delays exceeding 3 business days (non-island) typically result in a 2,000 won coupon compensation. For significant delays or damaged items, replacements and larger coupons have been issued.", + "actual_answer": "컨텍스트 내에서는 배송 지연에 대한 보상 사례에 대한 구체적인 언급이 없습니다. 따라서, 배송 지연에 대해 보상받은 사례가 있는지 여부는 제공된 자료에서 확인할 수 없습니다. 추가 문의나 구체적인 사례 요청이 필요할 것 같습니다.", + "elapsed_seconds": 0.8525848388671875, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 2, + "comment": "" + }, + "context_fit": { + "score": 1, + "comment": "" + }, + "clarity": { + "score": 1, + "comment": "컨텍스트 드러냄,제공된 자료 존재 드러냄, 추가 구체적인 확인 방향(일대일문의, 상담원 연결) 안내 없음" + } + }, + "manual_summary": { + "overall_score": 10, + "comment": "" + } + }, + { + "id": "Q037", + "tier": "hard", + "wall_type": "memory", + "question_ko": "저번에 물어본 배송 건 어떻게 됐어요?", + "question_en": "What happened to the shipping issue I asked about last time?", + "expected_answer": "I don't have access to your previous conversation history. Please provide your order number and I can look up the current status of your shipment directly.", + "actual_answer": "저번에 문의하신 배송 건에 대해 구체적인 내용이 컨텍스트에 포함되어 있지 않아 정확한 진행 상황을 알려드리기 어렵습니다. 주문 번호 또는 상세 내용을 알려주시면 더 도움을 드릴 수 있습니다.", + "elapsed_seconds": 1.0794711112976074, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 2, + "comment": "" + }, + "context_fit": { + "score": 2, + "comment": "" + }, + "clarity": { + "score": 1, + "comment": "컨텍스트 드러냄,제공된 자료 존재 드러냄, 추가 구체적인 확인 방향(일대일문의, 상담원 연결) 안내 없음" + } + }, + "manual_summary": { + "overall_score": 11, + "comment": "" + } + }, + { + "id": "Q038", + "tier": "hard", + "wall_type": "conditional", + "question_ko": "냉장 상품 오늘 금요일에 주문하면 언제 와요?", + "question_en": "If I order a cold-chain product today (Friday), when will it arrive?", + "expected_answer": "Cold-chain delivery is only available Monday through Thursday. An order placed on Friday will be dispatched the following Monday. Add 1-2 business days for delivery, so the earliest expected arrival would be Tuesday or Wednesday of next week.", + "actual_answer": "금요일에 주문하신 냉장 상품은 배송이 불가능합니다. 냉장 상품은 월요일부터 목요일까지만 배송이 가능하며, 금요일 주문 시에는 다음 주 월요일에 출고됩니다. 따라서, 금요일에 주문하시면 다음 주 월요일에 출고되어 배송이 시작됩니다.", + "elapsed_seconds": 0.9136950969696045, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "마치 아예 배송이 불가능하다는듯한 애매한 표현" + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q045", + "tier": "hard", + "wall_type": "contradiction", + "question_ko": "VIP 30일 반품 가능하다고 들었는데요", + "question_en": "I heard VIP members can return items within 30 days — is that true?", + "expected_answer": "The standard return window for all members including VIP is 14 calendar days. However, in certain exceptional cases (documented absence during return window, defect discovered after extended use), VIP members may be granted an extended return window at CS agent discretion. This is not a publicized policy and is applied case-by-case.", + "actual_answer": "VIP 회원님께서는 30일 반품이 가능하다는 내부 기준이 적용될 수 있습니다. 다만, 이 내용은 내부 가이드라인으로, 고객에게 공식적으로 안내하는 정책은 아닙니다. 만약 고객님께서 VIP 30일 반품 가능 여부를 문의하셨다면, 내부 규정에 따라 VIP 회원님께는 특별히 30일 반품이 가능할 수 있음을 안내하되, 구체적인 상황이나 상품에 따라 다를 수 있으니 고객센터에 문의하시거나, 고객님의 반품 요청 시 내부 기록에 \"VIP-EXTENDED\" 사유로 로그 기록이 필요하다는 점 참고 부탁드립니다.", + "elapsed_seconds": 1.4524881839752197, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 2, + "comment": "내부 문서 존재와 그 내용 드러냄" + }, + "context_fit": { + "score": 2, + "comment": "" + }, + "clarity": { + "score": 1, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 11, + "comment": "" + } + }, + { + "id": "Q050", + "tier": "medium", + "wall_type": null, + "question_ko": "반품 신청 후 환불받기까지 가장 빠른 경우 얼마나 걸려요?", + "question_en": "What is the fastest possible refund timeline after initiating a return?", + "expected_answer": "The fastest scenario: courier picks up the item, inspection completes in 1 business day, and Kakao Pay or Naver Pay refund is processed instantly. Minimum total time is approximately 2-3 business days from return request to refund. Credit card refunds take an additional 3-5 business days after inspection.", + "actual_answer": "반품 신청 후 환불받기까지 걸리는 시간은 결제 수단에 따라 다릅니다. \n- 신용카드(국내): 검수 후 3~5 영업일이 소요됩니다. \n- 카카오페이 또는 네이버페이: 검수 후 즉시 처리됩니다. \n- 계좌이체(은행 송금): 최대 7일이 걸릴 수 있습니다. \n- 해외 발급 카드(비자/마스터카드): 7~14일이 소요됩니다. \n\n환불이 완료되기까지는 보통 1~2 영업일의 검수 기간이 포함됩니다.", + "elapsed_seconds": 1.4143438339233398, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 3, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 15, + "comment": "" + } + }, + { + "id": "Q058", + "tier": "easy", + "wall_type": null, + "question_ko": "어떤 결제 수단을 지원하나요?", + "question_en": "What payment methods are supported?", + "expected_answer": "Supported payment methods include credit/debit cards, Kakao Pay, Naver Pay, real-time bank transfer, loyalty points, and discount coupons. Points and coupons can be combined with other payment methods. Virtual account (가상계좌) is not supported.", + "actual_answer": "저희는 신용카드(Visa, Mastercard, BC Card, NH Card), 디지털 월렛(Kakao Pay, Naver Pay, Toss Pay, Apple Pay), Toss Payments를 통한 계좌이체, 그리고 저희 자체 지갑인 Cholog Pay를 결제 수단으로 지원합니다. 기프트 카드와 적립금도 다른 결제 수단과 함께 사용할 수 있습니다. 다만, American Express와 Discover/Diners Club 카드는 지원하지 않습니다.", + "elapsed_seconds": 1.4416477680206299, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 3, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 15, + "comment": "" + } + }, + { + "id": "Q085", + "tier": "medium", + "wall_type": "noise", + "question_ko": "준비중 상태에서 취소는 즉시 처리되나요?", + "question_en": "Is cancellation immediate when the order is in Preparing status?", + "expected_answer": "Yes. Canceling an order while it is in Preparing status is processed instantly with no fee. Some agents have mistakenly advised that cancellation takes 1-2 business days — this is incorrect. You can cancel directly in the app and the status updates immediately.", + "actual_answer": "준비중 상태에서 주문을 취소하는 경우, 취소는 즉시 처리됩니다. 전체 금액이 원래 결제 수단으로 환불되며, 주문 상태가 \"Preparing\"인 동안에는 취소가 가능합니다.", + "elapsed_seconds": 1.1221091747283936, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "같은 내용 반복. 영어 안내" + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q088", + "tier": "hard", + "wall_type": "noise", + "question_ko": "VIP 포인트가 7%라고 어디선가 봤는데 맞나요?", + "question_en": "I saw somewhere that VIP members earn 7% loyalty points — is that accurate?", + "expected_answer": "No. The current VIP earning rate is 5%. The previous rate was 7% under the old policy (Standard 3%, Plus 5%, VIP 7%), but this was updated in early 2024. Current rates are Standard 1%, Plus 3%, VIP 5%. Some older documents or chat logs may still reference the outdated 7% figure.", + "actual_answer": "사용자님께서 말씀하신 VIP 포인트 적립률 7%는 현재 공식 FAQ와 정책에 명시되어 있지 않습니다. 현재 VIP 등급 고객의 포인트 적립률은 5%입니다. 따라서, 7%라는 비율은 맞지 않습니다.", + "elapsed_seconds": 5.205679893493652, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 2, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "아뇨 5%라고 말하면됨" + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + }, + { + "id": "Q097", + "tier": "hard", + "wall_type": "versioning", + "question_ko": "반품 기간이 10일이라고 들었는데 맞나요?", + "question_en": "I heard the return window is 10 days — is that right?", + "expected_answer": "No. The current return window is 14 calendar days (policy v3, effective April 2024). A 10-day window existed under policy v2 and a 7-day window under policy v1, but neither is in effect. If you encountered this information in an older document or a chat transcript, it is outdated.", + "actual_answer": "반품 기간은 제품에 따라 다를 수 있는데요, 일반적으로는 배송일로부터 14일 이내에 반품이 가능합니다. 다만, 일부 상품은 반품이 불가하거나 기간이 다를 수 있으니, 구체적인 상품 정보를 알려주시면 더 정확한 안내를 드릴 수 있습니다.", + "elapsed_seconds": 0.9053688049316406, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "10일 맞냐고 물었으면 먼저 아니 14일임을 명시하고 일부 상품마다 다를 수 있다고 안내하는게 맞지" + } + }, + "manual_summary": { + "overall_score": 14, + "comment": "" + } + }, + { + "id": "Q116", + "tier": "medium", + "wall_type": "cross_language", + "question_ko": "환불어케함? 반품하고싶은데", + "question_en": "How do I return something?", + "expected_answer": "Go to My Orders in the app or website, tap the order, select Request Return, choose a reason, and upload a photo. Courier pickup is free for defect returns; a 3,000 won fee applies for non-defect returns.", + "actual_answer": "반품을 원하시면, 먼저 상품이 반품 가능 기간인 14일 이내인지 확인해 주세요. 상품이 개봉되지 않았거나, 개봉했더라도 사용하지 않았으며 원래 포장과 모든 액세서리, 문서가 포함되어 있어야 합니다. \n\n반품 사유가 단순 변심이나 사이즈 변경인 경우, 3,000원의 처리비용이 공제될 수 있습니다. 반품 절차를 진행하려면 고객센터 또는 주문 내역에서 반품 요청을 접수하시면 됩니다. \n\n반품 후 환불은 결제 방법에 따라 3~5영업일 내에 처리됩니다. 자세한 안내는 고객센터에 문의하시거나, 주문 내역에서 반품 신청을 진행하시면 됩니다.", + "elapsed_seconds": 1.511747121810913, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 2, + "comment": "무조건 반품 가능 기간 14일 아님. 3~5영업일 아닌 결제수단도 있음" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 3, + "comment": "" + }, + "clarity": { + "score": 2, + "comment": "주문 내역에서 반품 신청하는 법이 더 자세히 들어있으면" + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + }, + { + "id": "Q150", + "tier": "medium", + "wall_type": "cross_language", + "question_ko": "불량품받았는데ㅠ 어케함 환불됨?", + "question_en": "I received a defective product what should I do?", + "expected_answer": "Request a return through My Orders and select Defective/Wrong Item as the reason. Upload photos of the defect. Courier pickup is free for defect returns and a full refund will be issued.", + "actual_answer": "불량품을 받으셨군요. 환불을 원하시면, 먼저 상품 검사가 완료된 후에 환불이 진행됩니다. 환불 소요 기간은 결제 방법에 따라 다르며, 카드 결제는 3~5일, 계좌이체는 최대 7일, 카카오페이 또는 네이버페이는 즉시 처리됩니다. 자세한 진행 상황은 고객센터에 문의하시면 도와드리겠습니다.", + "elapsed_seconds": 0.9752209186553955, + "request_error": false, + "auto_evaluation": { + "response_speed": { + "score": 3, + "criteria": "상=5초 이내, 중=5초 초과 10초 이내, 하=10초 초과" + } + }, + "manual_evaluation": { + "fact_accuracy": { + "score": 3, + "comment": "" + }, + "security": { + "score": 3, + "comment": "" + }, + "context_fit": { + "score": 2, + "comment": "불량품에 대한 자세한 안내 부족함" + }, + "clarity": { + "score": 2, + "comment": "" + } + }, + "manual_summary": { + "overall_score": 13, + "comment": "" + } + } + ] +} diff --git a/mission/wall-report.md b/mission/wall-report.md index 4994eca..e127aca 100644 --- a/mission/wall-report.md +++ b/mission/wall-report.md @@ -7,26 +7,34 @@ > 구현하면서 잘 안 됐던 것, 예상과 달랐던 것을 적어주세요. -- - +- 초반 조회 결과에 질문 관련 내용이 없길래 topK 값을 2배~4배까지 늘려 해당 질문에 대한 답변을 잘할 수 있도록 개선했습니다. +하지만 그 외 다른 질문에 대한 정확도는 확연히 떨어지는 것을 발견했습니다. 컨텍스트에 많은 내용이 들어가 오히려 혼란을 준 것 같습니다. +현재 구현 기준 기본값인 topK=4가 가장 최적으로 보여 롤백했습니다. +- 네이버페이에 대해 물어봤는데 카카오페이에 대한 답변을 하는 경우가 있었습니다. +알고보니 ChatMemory 의해 이전 질문이 컨텍스트에 포함되며 이전 질문을 현재 질문이라고 착각해서 생긴 문제였습니다. +ChatMemory를 제거하고 대화 내역이 보존되지 않게 했지만 추후 해결해야할 문제입니다. ## 2. 해결하지 못한 것 > 시도했지만 결국 해결 못한 문제가 있다면 적어주세요. -- - +- 컨텍스트에 질문에 대한 정답이 포함되어 있는데 틀린 답변을 내놓습니다. 관련 케이스를 살펴보니 정답 내용이 첫번째에 오지 않을 경우 주로 발생했습니다. + 해결하기 위해 리랭크 개념을 학습하고 llm에게 청크 재정렬을 맡겨봤는데 이 또한 제가 의도한대로 재정렬되지 않았습니다. +- 컨텍스트에 상반된 내용이 있을 때에 따라 다른 대답을 하는 문제를 해결하지 못했습니다. 예를 들어 `VIP는 냉장 배송도 무료배송인가요?`를 물었을 때 +`vip는 모두 무료다`와 `냉장 배송은 등급 상관없이 4000원 고정이다`라는 내용이 섞여 `vip는 냉장 배송도 무료다` 라는 잘못된 답변을 내놓습니다. +프롬프트에 내용이 상반될 경우 더 상세한 것, 세부적인 것을 기준으로 하라고 명시했지만 상세하고 세부적인 것의 기준 조차 애매해 잘동작하지 않습니다. +- 명확하지 않은 단어를 잘 이해하지 못하는 경우가 많습니다. 예를 들어 `비구독 상품`이라고 하면 비구독 사용자가 상품을 구매하는 경우라고 잘못 이해해 이해해 + 틀린 답변을 내뱉습니다. ## 3. 정확도 측정 결과 -> 테스트 질문 100개로 측정한 정확도를 기록해주세요. - -| 난이도 | 정확도 | 비고 | -|--------|--------|------| -| easy | | | -| medium | | | -| hard | | | +> 테스트 질문 150개로 측정한 정확도를 기록해주세요. +| 난이도 | 정확도 | 비고 | +|--------|-------|----| +| easy | 14/30 | | +| medium | 27/94 | | +| hard | 6/26 | | ## 4. 왜 그런 결과가 나왔는지 @@ -34,9 +42,16 @@ - - ## 5. 개선하고 싶은 것 > 시간이 더 있었다면 시도해보고 싶은 개선점을 적어주세요. -- +- 답변의 말투를 개선하고 싶습니다. 현재는 같은 내용을 반복하거나, 사용자가 응?할만한 말투를 구사합니다. + 또한 내부 컨텍스트 문서의 존재를 답변에 포함하기도 합니다. +- 대화 맥락을 보존해 더욱 고객 응대에 능한 챗봇을 만들고 싶습니다. +- 인메모리 외 인프라를 구축해 정말 실무의 챗봇과 유사하게 만들어보고 싶습니다. +- 평가 방식을 개선해 모델의 성능을 제대로 측정하고 반복 개선해나가보고 싶습니다. 현재 평가는 이걸 왜 틀렸다고 하는거지?싶은 부분이 많습니다. + 챗봇의 목적마다 평가 방식이 다르니 제가 만들고자 하는 챗봇을 명확히 정의하고 그에 맞는 평가 방식을 새로 만들어보고 싶습니다. +- 챗 로그, 이력을 관리하고 활용해보고 싶습니다. 주어진 데이터셋의 챗 로그는 컨텍스트에 넣지 않고 가장 많이 묻는 질문과 같은 운영 통계 목적으로 활용했습니다. + 다른 방향으로 활용할 수 있다면 참고해 챗 로그까지 활용해 데이터 축적에 따른 성능 향상을 이루고 싶습니다. +- 구버전 문서도 마찬가지로 필요 없다고 느껴 제거했는데 필요하다면 포함해 응답 수준을 높여보고 싶습니다. diff --git a/src/main/java/com/cholog/bootcamp/config/MarkdownReader.java b/src/main/java/com/cholog/bootcamp/config/MarkdownReader.java new file mode 100644 index 0000000..33c7619 --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/config/MarkdownReader.java @@ -0,0 +1,48 @@ +package com.cholog.bootcamp.config; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.ai.document.Document; +import org.springframework.ai.reader.markdown.MarkdownDocumentReader; +import org.springframework.ai.reader.markdown.config.MarkdownDocumentReaderConfig; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.core.io.Resource; +import org.springframework.stereotype.Component; + +import lombok.extern.slf4j.Slf4j; + +@Slf4j +@Component +public class MarkdownReader { + + private final Resource[] resources; + + public MarkdownReader(@Value("classpath:data/**/*.md") Resource[] resources) { + this.resources = resources; + } + + public List loadAll() { + List allDocuments = new ArrayList<>(); + for (Resource resource : resources) { + MarkdownDocumentReaderConfig config = MarkdownDocumentReaderConfig.builder() + .withHorizontalRuleCreateDocument(true) + .withIncludeCodeBlock(false) + .withIncludeBlockquote(false) + .withAdditionalMetadata("filename", resource.getFilename()) + .build(); + + MarkdownDocumentReader reader = new MarkdownDocumentReader(resource, config); + allDocuments.addAll(reader.get()); + } + + allDocuments.forEach(doc -> log.info( + "filename={}, title={}, text={}", + doc.getMetadata().get("filename"), + doc.getMetadata().get("title"), + doc.getText() + )); + + return allDocuments; + } +} diff --git a/src/main/java/com/cholog/bootcamp/config/SimpleVectorStoreConfig.java b/src/main/java/com/cholog/bootcamp/config/SimpleVectorStoreConfig.java new file mode 100644 index 0000000..ba597d1 --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/config/SimpleVectorStoreConfig.java @@ -0,0 +1,73 @@ +package com.cholog.bootcamp.config; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.List; + +import org.springframework.ai.embedding.EmbeddingModel; +import org.springframework.ai.vectorstore.SimpleVectorStore; +import org.springframework.boot.ApplicationRunner; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration +public class SimpleVectorStoreConfig { + + private static final String MODE_OPTION = "mode"; + private static final String EMBEDDING_MODE = "embedding"; + + @Bean + public SimpleVectorStore vectorStore(EmbeddingModel embeddingModel) { + return SimpleVectorStore.builder(embeddingModel).build(); + } + + /** + * VectorStore 임베딩 runner + * 애플리케이션 실행 시 --mode=embedding 옵션이 존재하면 임베딩을 새로 수행한다. + * + * bootrun 실행 시 + * `./gradlew bootrun --args='--mode=embedding' + * + * jar 실행 시 + * `java -jar --mode=embedding` + */ + @Bean + public ApplicationRunner applicationRunner(SimpleVectorStore vectorStore, MarkdownReader markdownReader) { + File vectorFile = Paths.get("data", "vector-store.json").toFile(); + createNewVectorFile(vectorFile); + + return args -> { + List options = args.getOptionValues(MODE_OPTION); + if (options == null || options.isEmpty()) { + vectorStore.load(vectorFile); + return; + } + + for (String option : options) { + if (option.equals(EMBEDDING_MODE)) { + doEmbedding(vectorStore, markdownReader, vectorFile); + return; + } + } + + vectorStore.load(vectorFile); + }; + } + + private void doEmbedding(SimpleVectorStore vectorStore, MarkdownReader markdownReader, File vectorFile) { + vectorFile.delete(); + createNewVectorFile(vectorFile); + vectorStore.doAdd(markdownReader.loadAll()); + vectorStore.save(vectorFile); + } + + private void createNewVectorFile(File vectorFile) { + try { + vectorFile.createNewFile(); + } catch (IOException e) { + throw new RuntimeException("vector-store.json 생성 실패", e); + } + } +} diff --git a/src/main/java/com/cholog/bootcamp/controller/ChatPageController.java b/src/main/java/com/cholog/bootcamp/controller/ChatPageController.java new file mode 100644 index 0000000..c70e726 --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/controller/ChatPageController.java @@ -0,0 +1,35 @@ +package com.cholog.bootcamp.controller; + +import java.util.List; + +import org.springframework.stereotype.Controller; +import org.springframework.ui.Model; +import org.springframework.web.bind.annotation.GetMapping; + +@Controller +public class ChatPageController { + + @GetMapping({"/", "/chat"}) + public String chat(Model model) { + model.addAttribute("pageTitle", "초록 고객지원 챗봇"); + model.addAttribute("initialMessage", """ + 안녕하세요. 초록 고객지원 챗봇입니다.🤖 + 무엇을 도와드릴까요? + + 📞 운영시간 안내 + 자동 챗봇은 24시간 이용하실 수 있습니다. + 상담사 연결 및 전화 상담은 평일 오전 9시부터 오후 6시까지 가능하며, 주말 및 공휴일에는 운영되지 않습니다. + 운영 시간 외 문의는 챗봇을 이용하시거나 이메일로 남겨주시면 다음 영업일부터 순차적으로 확인해 드리겠습니다. + + ☎️ 문의 전화: 1588-0000 + 📧 이메일: support@cholog.kr + 💬 카카오톡: @초록 + """); + model.addAttribute("quickPrompts", List.of( + "배송은 보통 얼마나 걸리나요?", + "반품 신청 기준을 알려주세요.", + "멤버십 등급 혜택이 궁금해요." + )); + return "chat"; + } +} diff --git a/src/main/java/com/cholog/bootcamp/controller/ChatbotController.java b/src/main/java/com/cholog/bootcamp/controller/ChatbotController.java new file mode 100644 index 0000000..b2a361b --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/controller/ChatbotController.java @@ -0,0 +1,31 @@ +package com.cholog.bootcamp.controller; + +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestBody; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; + +import com.cholog.bootcamp.dto.ChatbotRequest; +import com.cholog.bootcamp.dto.ChatbotResponse; +import com.cholog.bootcamp.service.ChatbotService; + +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; + +@Slf4j +@RequiredArgsConstructor +@RequestMapping("/api/chat") +@RestController +public class ChatbotController { + + private final ChatbotService chatbotService; + + @PostMapping + public ResponseEntity chat( + @RequestBody ChatbotRequest request + ) { + ChatbotResponse response = chatbotService.chat(request); + return ResponseEntity.ok().body(response); + } +} diff --git a/src/main/java/com/cholog/bootcamp/dto/ChatbotRequest.java b/src/main/java/com/cholog/bootcamp/dto/ChatbotRequest.java new file mode 100644 index 0000000..0566ce3 --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/dto/ChatbotRequest.java @@ -0,0 +1,6 @@ +package com.cholog.bootcamp.dto; + +public record ChatbotRequest( + String question +) { +} diff --git a/src/main/java/com/cholog/bootcamp/dto/ChatbotResponse.java b/src/main/java/com/cholog/bootcamp/dto/ChatbotResponse.java new file mode 100644 index 0000000..edc1e91 --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/dto/ChatbotResponse.java @@ -0,0 +1,27 @@ +package com.cholog.bootcamp.dto; + +import org.springframework.ai.chat.metadata.Usage; + +public record ChatbotResponse( + String answer, + TokenUsageInfo tokenUsage +) { + + public static ChatbotResponse from(String answer, Usage usage) { + return new ChatbotResponse( + answer, + new TokenUsageInfo( + usage.getPromptTokens(), + usage.getCompletionTokens(), + usage.getTotalTokens() + ) + ); + } + + private record TokenUsageInfo( + int promptTokens, + int completionTokens, + int totalTokens + ) { + } +} diff --git a/src/main/java/com/cholog/bootcamp/service/ChatbotService.java b/src/main/java/com/cholog/bootcamp/service/ChatbotService.java new file mode 100644 index 0000000..03b12fa --- /dev/null +++ b/src/main/java/com/cholog/bootcamp/service/ChatbotService.java @@ -0,0 +1,121 @@ +package com.cholog.bootcamp.service; + +import java.io.IOException; +import java.util.List; +import java.util.stream.Collectors; + +import org.springframework.ai.chat.client.ChatClient; +import org.springframework.ai.chat.metadata.Usage; +import org.springframework.ai.chat.model.ChatResponse; +import org.springframework.ai.document.Document; +import org.springframework.ai.reader.TextReader; +import org.springframework.ai.vectorstore.SearchRequest; +import org.springframework.ai.vectorstore.VectorStore; +import org.springframework.core.io.support.PathMatchingResourcePatternResolver; +import org.springframework.core.io.support.ResourcePatternResolver; +import org.springframework.stereotype.Service; + +import com.cholog.bootcamp.dto.ChatbotRequest; +import com.cholog.bootcamp.dto.ChatbotResponse; + +import lombok.extern.slf4j.Slf4j; + +@Slf4j +@Service +public class ChatbotService { + + private final ChatClient chatClient; + private final VectorStore vectorStore; + private final ResourcePatternResolver resolver; + + public ChatbotService( + VectorStore vectorStore, + ChatClient.Builder chatClientBuilder + ) { + this.chatClient = chatClientBuilder.build(); + this.vectorStore = vectorStore; + this.resolver = new PathMatchingResourcePatternResolver(); + } + + public ChatbotResponse chat(ChatbotRequest request) { + // 검색 + SearchRequest searchRequest = getSearchRequest(request.question(), 4); + List documents = vectorStore.similaritySearch(searchRequest); + + // 증강 & 생성 + documents = getFullDocuments(documents); + String context = getContext(documents); + ChatResponse chatResponse = chatClient.prompt() + .system(""" + 당신은 초록 고객센터의 챗봇입니다. + [답변 규칙]을 준수하며 주어진 [컨텍스트]를 기반으로 [사용자 질문]에 답변해주세요. + + [답변 규칙] + - 모든 답변에는 `내부 문서`나 `컨텍스트`를 언급하지 말아주세요. + - 모든 답변은 제공된 컨텍스트를 기반으로만 하세요. 절대 일반 상식으로 추론하지 마세요. + - [답변 불가 유형]의 질문일 경우 답변할 수 없음을 안내하고 [추가 문의 안내]를 그대로 출력하세요. + - 내용이 충돌하는 경우 다음 우선순위를 따라 답변 합니다. + - 질문 도메인과 가장 근접한 내용 + - 더 구체적인 상황을 다루는 내용 + - 더 최신 버전의 내용 + + [답변 불가 유형] + - 컨텍스트에 관련 내용이 없는 질문 + - 개인 정보가 요구되는 질문 + + [추가 문의 안내] + ``` + 📝 문의 접수 + 앱/웹 고객센터 > 문의하기에서 문의를 남겨주세요. + + 💬 상담원 연결 + 실시간 채팅: 매일 08:00 ~ 22:00 + 전화 상담: 평일 09:00 ~ 18:00 + 일반 고객센터: 1588-0000 + VIP 전용 상담: 1588-0002 + ``` + """) + .user(""" + [사용자 질문] + %s + + [컨텍스트] + %s + """.formatted(request.question(), context)) + .call() + .chatResponse(); + + String answer = chatResponse.getResult().getOutput().getText(); + Usage usage = chatResponse.getMetadata().getUsage(); + return ChatbotResponse.from(answer, usage); + } + + private List getFullDocuments(List documents) { + return documents.stream() + .map(document -> document.getMetadata().get("filename").toString()) + .distinct() + .map(filename -> { + try { + return resolver.getResources("classpath:data/**/" + filename)[0]; + } catch (IOException e) { + throw new RuntimeException(e); + } + }) + .map(TextReader::new) + .flatMap(reader -> reader.get().stream()) + .toList(); + } + + private static String getContext(List documents) { + return documents.stream() + .map(Document::getText) + .collect(Collectors.joining("\n\n")); + } + + private SearchRequest getSearchRequest(String query, int k) { + return SearchRequest.builder() + .query(query) + .topK(k) + .build(); + } +} diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml index 6233b35..6793d53 100644 --- a/src/main/resources/application.yml +++ b/src/main/resources/application.yml @@ -1,4 +1,6 @@ spring: + config: + import: optional:file:.env[.properties] # 루트의 .env 파일이 존재하면 .properties 형식으로 읽어 임포트 application: name: spring-ai-bootcamp-basic ai: diff --git a/src/main/resources/data/layer1_faq/account.md b/src/main/resources/data/layer1_faq/account.md new file mode 100644 index 0000000..8762172 --- /dev/null +++ b/src/main/resources/data/layer1_faq/account.md @@ -0,0 +1,45 @@ +# Account & Membership + +## Cholog Corporation — Official FAQ + +### What are the membership tiers and their benefits? + +| Tier | Qualification | Free Shipping | Points | Extras | +|------|--------------|---------------|--------|--------| +| Standard | Default | Economy on orders ≥ 20,000 won | 1% | — | +| Plus | Spend ≥ 200,000 won/year | Economy on all orders | 3% | Birthday double-points | +| VIP | Spend ≥ 800,000 won/year | Priority on all orders | 5% | Early launch access, dedicated support line (1588-0002) | + +Tier status is evaluated on January 1st each year based on the prior year's spending. +Returned items are deducted from annual spending totals and may trigger a tier downgrade. + +### How does the loyalty point system work? + +Points are earned at 1% (Standard), 3% (Plus), or 5% (VIP) of the pre-shipping order total. +Points are credited 7 days after delivery confirmation and expire 12 months from the date +earned. 1 point = 1 won. Minimum redemption: 1,000 points per order. + +### How do I create an account? + +You need a valid email address or Korean mobile number. You can also register instantly via +Kakao, Naver, or Google SSO. All new accounts start at Standard tier and receive a one-time +3,000 won welcome coupon valid for 30 days. + +### How do I permanently delete my account? + +Go to "Settings > Privacy > Delete My Account". After confirming with your password or +a one-time verification code, deletion is scheduled within 7 business days. All personal +data, order history, saved addresses, and payment tokens are purged under Korea's Personal +Information Protection Act (PIPA). Unused points and coupons are forfeited upon deletion. + +### How do I enable two-factor authentication (2FA)? + +Go to "Settings > Security > Two-Factor Authentication". Choose SMS or an authenticator +app (Google Authenticator, Authy). Once enabled, a 6-digit code is required at every login. +Recovery codes are provided during setup — store them safely. + +### When do loyalty points expire? + +Points expire 12 months from the date they are earned. If you return an order, the points +earned from that order are deducted within 24 hours of the refund. If your balance is +insufficient, the deficit is carried forward and deducted from future earnings. diff --git a/src/main/resources/data/layer1_faq/eco_green.md b/src/main/resources/data/layer1_faq/eco_green.md new file mode 100644 index 0000000..f61d191 --- /dev/null +++ b/src/main/resources/data/layer1_faq/eco_green.md @@ -0,0 +1,38 @@ +# Environment & Green Initiatives + +## Cholog Corporation — Official FAQ + +### What is the Eco Choice certification? + +Products bearing the green leaf "Eco Choice" badge meet at least one of: made from +recycled materials, certified by Korea Eco-Label (환경부 인증), carbon-neutral shipping, +or biodegradable packaging. Filter by "Eco Choice" in any product category to browse. +Eco Choice status is reviewed annually and may be revoked if the product no longer meets criteria. + +### How does the Green Packaging option work? + +Select "Green Packaging" at checkout to receive your order in minimal, fully recyclable +packaging with no plastic fillers. Green Packaging is free and reduces box size by up to +40%. Each order using Green Packaging earns a 200-point eco bonus credited 7 days after +delivery confirmation. + +### What is the Carbon Points program? + +Every completed order earns 1 Carbon Point per 1,000 won spent. Carbon Points accumulate +in your account and can be donated to verified reforestation projects (1 Carbon Point = +1 tree planted) or converted to 100 loyalty points each. Carbon Points do not expire +and are separate from loyalty points. + +### How does the electronics take-back program work? + +Bring used electronics to any CU convenience store collection point. You receive 2,000 +loyalty points per item dropped off. For large items (TVs, appliances), free pickup can +be scheduled through "Help Center > Electronics Recycling". Items are recycled through +Korea's certified e-waste partners. + +### What eco-friendly packaging materials does Cholog use? + +Standard boxes are made from 80%+ recycled cardboard. Air pillows use plant-based +biodegradable film. Tape is paper-based. The Green Packaging option eliminates all +non-essential filler materials. Cholog aims to achieve 100% plastic-free packaging +by end of 2026. diff --git a/src/main/resources/data/layer1_faq/feedback.md b/src/main/resources/data/layer1_faq/feedback.md new file mode 100644 index 0000000..6cf9f33 --- /dev/null +++ b/src/main/resources/data/layer1_faq/feedback.md @@ -0,0 +1,29 @@ +# Complaints & Feedback + +## Cholog Corporation — Official FAQ + +### How do I file a complaint? + +Go to "Help Center > Submit Feedback" on our website or app. Select the category (product +quality, delivery issue, website bug, suggestion, etc.) and describe your experience. +Complaints receive an acknowledgment email within 2 hours and a full response from our +customer experience team within 48 hours. + +### Can I leave a product review? + +Yes. Only customers who have purchased and received the product can leave a review — +labeled "Verified Purchase". Photo and video reviews earn 500 bonus points. Reviews that +violate community guidelines (spam, offensive content) are removed by our moderation team. + +### How do I report a counterfeit or unsafe product? + +Tap "Report" on the product page and select "Counterfeit Suspected" or "Safety Concern". +Our quality assurance team investigates within 48 hours. If confirmed, the product is +removed, the seller is penalized, and all affected buyers receive a full refund plus a +10,000 won inconvenience coupon. + +### What is the Cholog Community Forum? + +The Community Forum at community.cholog.kr is a space for customers to ask questions, +share tips, and discuss products. Staff moderators answer questions tagged "@staff". +Active contributors earn Community Points exchangeable for coupons. Available in Korean only. diff --git a/src/main/resources/data/layer1_faq/invoice.md b/src/main/resources/data/layer1_faq/invoice.md new file mode 100644 index 0000000..4285268 --- /dev/null +++ b/src/main/resources/data/layer1_faq/invoice.md @@ -0,0 +1,26 @@ +# Tax Invoice & Receipts + +## Cholog Corporation — Official FAQ + +### How do I get a tax invoice (세금계산서)? + +A digital receipt is automatically emailed after every purchase. For a formal tax invoice, +go to "My Orders", select the order, and tap "Request Tax Invoice". Enter your business +registration number (사업자등록번호). Tax invoices are issued within 1 business day and +sent to your registered email address. + +Tax invoice requests must be made within the same calendar month as the purchase. +Requests for prior months are not accepted. + +### Can I get a receipt for a corporate expense report? + +Yes. The automatically issued digital receipt serves as a valid purchase receipt. +For businesses requiring a tax invoice with VAT breakdown, follow the tax invoice +request process above. Business accounts registered at b2b.cholog.kr can request +monthly consolidated invoices covering all orders in the billing period. + +### What if I entered the wrong business registration number? + +Contact customer support within 1 business day of the original request. We can void +and reissue a tax invoice with the correct number within the same calendar month. +After the month closes, corrections are not possible under Korean tax law. diff --git a/src/main/resources/data/layer1_faq/marketplace.md b/src/main/resources/data/layer1_faq/marketplace.md new file mode 100644 index 0000000..ad2d9c9 --- /dev/null +++ b/src/main/resources/data/layer1_faq/marketplace.md @@ -0,0 +1,32 @@ +# Cholog Marketplace + +## Cholog Corporation — Official FAQ + +### What is Cholog Marketplace? + +Cholog Marketplace is our platform for verified third-party sellers. Marketplace products +are clearly labeled with the seller's name on the product page and order confirmation. +Cholog guarantees buyer protection on all Marketplace transactions up to 500,000 won. +Beyond that amount, disputes are handled directly between buyer and seller. + +### Does the standard Cholog return policy apply to Marketplace items? + +No. Marketplace items follow the individual seller's return policy, which may differ +significantly from Cholog's standard 14-day window. Return windows for Marketplace +sellers range from 3 to 30 days depending on the seller. Always check the seller's +policy on the product page before purchasing. Cholog's buyer protection guarantee +covers cases of item not received or item significantly not as described. + +### How do I become a Marketplace seller? + +Apply at seller.cholog.kr with your business registration certificate, product catalog, +and bank account details. Review takes approximately 5 business days. Approved sellers +pay a commission of 5–12% per transaction depending on the product category. Sellers +must maintain a customer satisfaction rating above 4.2 / 5.0 to remain listed. + +### What if I have a dispute with a Marketplace seller? + +Contact the seller directly first through the in-app messaging feature ("My Orders > +Contact Seller"). If unresolved within 3 business days, you can escalate to Cholog +Buyer Protection by selecting "Open Dispute" in "My Orders". Cholog mediates and may +issue a full refund under the buyer protection guarantee within 5 business days. diff --git a/src/main/resources/data/layer1_faq/orders.md b/src/main/resources/data/layer1_faq/orders.md new file mode 100644 index 0000000..8064283 --- /dev/null +++ b/src/main/resources/data/layer1_faq/orders.md @@ -0,0 +1,49 @@ +# Orders & Order Management + +## Cholog Corporation — Official FAQ + +### How do I cancel an order? + +You can cancel an order by going to "My Orders" and tapping "Cancel Order" while the status +is "Preparing". Cancellation is instant and the full amount is refunded to your original +payment method. Once the status changes to "Shipped", cancellation is no longer possible — +you must wait for delivery and then initiate a return. + +Note: Marketplace seller orders may have a shorter cancellation window. Check the seller's +policy on the product page before placing the order. + +### Can I modify my order after placing it? + +You can change the delivery address only if the order status is still "Preparing". Go to +"My Orders", tap the order, and select "Edit Address". Quantity changes and product swaps +are not supported after checkout — cancel the order and place a new one instead. + +### How do I track my order? + +After dispatch, you receive a Hanjin Express tracking number via email and KakaoTalk. +Use the "Track Shipment" feature in the Cholog app or enter the number at hanjin.kr. +Real-time GPS tracking is available for Priority shipments only. + +### What is the cutoff time for same-day dispatch? + +Orders confirmed before 1 PM KST on business days are handed to the courier the same +afternoon. Orders placed after 1 PM, on weekends, or on public holidays enter the dispatch +queue for the next business day morning. + +### Can I combine multiple orders into one shipment? + +Orders placed within 30 minutes of each other and sharing the same delivery address may be +combined automatically. To request manual consolidation, contact customer support before +the first order enters "Shipped" status. Consolidation is not available for Priority or +same-day shipments. + +### What happens if my package is lost or damaged in transit? + +Go to "My Orders", select the affected order, and tap "Report Issue". Upload photos of +the damage or describe the missing items. Our logistics team investigates within 2 business +days and offers a replacement shipment or full refund. + +### Can I schedule delivery for a future date? + +Yes. At checkout, select "Scheduled Delivery" and choose a date up to 14 days in advance. +Available for Economy and Priority shipping tiers. Same-day delivery cannot be pre-scheduled. diff --git a/src/main/resources/data/layer1_faq/packaging.md b/src/main/resources/data/layer1_faq/packaging.md new file mode 100644 index 0000000..9f30805 --- /dev/null +++ b/src/main/resources/data/layer1_faq/packaging.md @@ -0,0 +1,31 @@ +# Special Packaging & Delivery Types + +## Cholog Corporation — Official FAQ + +### Do you ship fresh or frozen products? + +Yes. Fresh and frozen items are shipped in insulated packaging with ice packs via CJ Logistics +Fresh (dedicated cold-chain courier). Cold-chain delivery is available only Monday through +Thursday to prevent weekend transit degradation. A flat cold-chain fee of 4,000 won applies +regardless of order size. Cold-chain orders cannot be combined with standard shipments. + +### How are large or heavy items delivered? + +Items over 20 kg or classified as "Large Item" are delivered by a Lotte Global Logistics +two-person team. Delivery includes placement in the room of your choice (ground floor). +Buildings with restricted elevator access or stair-carry requirements incur an additional +fee of 15,000 won per floor. Schedule large-item delivery at checkout or by contacting +customer support. + +### What is the Green Packaging option? + +Select "Green Packaging" at checkout to receive your order in minimal, fully recyclable +packaging with no plastic fillers. Green Packaging is free of charge and reduces box size +by up to 40%, lowering carbon emissions. Each Green Packaging order earns a 200-point +eco bonus credited 7 days after delivery. + +### Is gift wrapping available? + +Yes. Select "Gift Wrap" at checkout for 1,500 won per item. Includes a branded box, +ribbon, and a personalized message card (up to 100 characters). Gift wrapping is not +available for oversized items or Marketplace seller products. diff --git a/src/main/resources/data/layer1_faq/payment.md b/src/main/resources/data/layer1_faq/payment.md new file mode 100644 index 0000000..2c4a6d9 --- /dev/null +++ b/src/main/resources/data/layer1_faq/payment.md @@ -0,0 +1,43 @@ +# Payment + +## Cholog Corporation — Official FAQ + +### Which payment methods do you accept? + +We accept: Visa, Mastercard, BC Card, NH Card (credit/debit), digital wallets (Kakao Pay, +Naver Pay, Toss Pay, Apple Pay), direct bank transfer via Toss Payments, and Cholog Pay +(our proprietary wallet). Gift cards and store credit can be combined with any method. + +Foreign cards (Visa/Mastercard issued abroad) are accepted and charged in KRW. +American Express and Discover/Diners Club are not accepted. + +### Can I split payment across multiple methods? + +Yes. At checkout, select "Split Payment" to combine up to two methods — for example, part +with Naver Pay and the remainder with a credit card. Gift card balances can also be combined +with any other method. Only one coupon can be applied per order. + +### Do you offer interest-free installment plans? + +Orders totaling 30,000 won or more are eligible for 2- or 3-month interest-free installments +on BC Card and NH Card. Orders over 100,000 won unlock a 6-month interest-free plan. +Installment options appear automatically at checkout when an eligible card is detected. + +### How do I get a tax invoice (세금계산서)? + +A digital receipt is sent automatically by email after every purchase. For a tax invoice, +go to "My Orders", select the order, and tap "Request Tax Invoice". Enter your business +registration number. Tax invoices are issued within 1 business day. + +### What happens if my payment is declined? + +The order is not placed and no charge is made. Common reasons: insufficient funds, daily +transaction limit, or an expired card. Verify your card details and try again, or switch +to an alternative payment method. Duplicate authorization holds disappear within 3–5 +business days. + +### What is Cholog Pay? + +Cholog Pay is our proprietary wallet. Load funds via bank transfer and earn a 2% top-up +bonus (load 10,000 won → receive 10,200 won). Balance can be used for any purchase and +is refundable minus the bonus amount. Cholog Pay cannot be used to purchase gift cards. diff --git a/src/main/resources/data/layer1_faq/returns.md b/src/main/resources/data/layer1_faq/returns.md new file mode 100644 index 0000000..08e4a53 --- /dev/null +++ b/src/main/resources/data/layer1_faq/returns.md @@ -0,0 +1,58 @@ +# Returns & Refunds + +## Cholog Corporation — Official FAQ + +### What is the return policy? + +Products may be returned within 14 calendar days of delivery. The item must be unopened or, +if opened, unused with all original accessories and packaging intact. The following cannot +be returned under any circumstances: Clearance items, digital downloads, personalized or +engraved products, opened cosmetics and skincare, underwear and swimwear, and items damaged +by the customer. + +Note: Marketplace seller items follow the individual seller's return policy, which may differ +from Cholog's standard 14-day window. + +### How do I initiate a return? + +Open the Cholog app or website, go to "My Orders", tap the order, and select "Request Return". +Choose a reason, upload at least one photo of the item, and select a pickup date. For +defect-related returns, courier pickup is free. For non-defect returns (change of mind, +wrong size), a 3,000 won handling fee is deducted from the refund. + +### How long does a refund take? + +After we receive and inspect the returned item (1–2 business days), we issue the refund: +- **Credit card**: 3–5 business days +- **Kakao Pay / Naver Pay**: typically instant +- **Bank transfer**: up to 7 business days +- **International card (Visa/Mastercard issued abroad)**: 7–14 business days + +### Can I exchange an item instead of returning it? + +Yes. When initiating a return, select "Exchange" instead of "Refund". Choose the replacement +item (same product, different size or color). If the replacement costs more, you pay the +difference; if less, the difference is refunded. Exchanges are processed within 3–5 business +days after pickup. + +### Are there items that cannot be returned? + +The following are final sale and non-refundable: +- Clearance / sale items +- Digital downloads +- Personalized / engraved items +- Opened cosmetics, skincare, underwear, swimwear +- Gift cards +- Items the customer has damaged + +### What if I received the wrong item or a defective product? + +Contact customer support or use "Report Issue" in "My Orders". We arrange a free pickup of +the incorrect item and ship the correct one via Priority at no charge. A 5,000 won apology +coupon is also issued automatically. + +### Can I return a promotional or discounted item? + +Yes. Promotional items follow the standard 14-day return policy. The refund amount is based +on the price actually paid, not the original price. For bundle deals, all items in the +bundle must be returned together for a full refund. diff --git a/src/main/resources/data/layer1_faq/shipping.md b/src/main/resources/data/layer1_faq/shipping.md new file mode 100644 index 0000000..0dd973b --- /dev/null +++ b/src/main/resources/data/layer1_faq/shipping.md @@ -0,0 +1,48 @@ +# Shipping & Delivery + +## Cholog Corporation — Official FAQ + +### What shipping options are available? + +We offer three shipping tiers: +- **Economy**: 2,500 won, 2–4 business days +- **Priority**: 5,000 won, guaranteed next-business-day delivery by noon +- **Same-Day**: 8,000 won, available in Seoul / Incheon / Gyeonggi for orders before 10 AM KST (weekdays only) + +Free Economy shipping applies to orders over 20,000 won for Standard members. +Gold-tier members receive free Priority shipping on every order. + +### Which courier delivers my order? + +All standard domestic orders are fulfilled through Hanjin Express. Cold-chain (fresh/frozen) +orders are shipped via CJ Logistics Fresh. Large-item deliveries (furniture, appliances over +20 kg) are handled by Lotte Global Logistics with a two-person team. + +### Do you deliver to Jeju Island or offshore islands? + +Yes. We deliver to Jeju, Ulleungdo, Geojedo, and all major offshore islands. Delivery takes +an additional 1–2 business days beyond the standard timeframe. There is no extra surcharge +for island delivery. + +### What happens if I am not home when the courier arrives? + +The courier makes two delivery attempts. After both fail, your package is held at the nearest +Hanjin Express depot for 7 calendar days. You can reschedule via the Hanjin app or pick it +up at the depot with a valid ID. + +### Can I pick up my order at a convenience store? + +Yes. Select "CU Convenience Store Pickup" at checkout and choose a CU location near you. +The package is held for 5 calendar days; uncollected parcels are automatically returned and +a full refund is issued. + +### Can I request a specific delivery time slot? + +Priority shipping orders allow you to select a 2-hour window between 9 AM and 9 PM. +Economy shipping does not support time-slot selection. Availability depends on your region +and courier capacity. + +### What is the maximum package size and weight? + +Individual packages cannot exceed 30 kg or 160 cm in combined dimensions. Orders exceeding +these limits are automatically split into multiple packages at no extra shipping cost. diff --git a/src/main/resources/data/layer1_faq/subscription.md b/src/main/resources/data/layer1_faq/subscription.md new file mode 100644 index 0000000..1afa6e1 --- /dev/null +++ b/src/main/resources/data/layer1_faq/subscription.md @@ -0,0 +1,40 @@ +# Subscription Services + +## Cholog Corporation — Official FAQ + +### What subscription plans are available? + +| Plan | Price | Frequency | Contents | +|------|-------|-----------|----------| +| Essentials Box | 29,900 won | Monthly | Curated household essentials | +| Coffee Club | 19,900 won | Bi-weekly | Specialty coffee beans | +| Beauty Box | 39,900 won | Monthly | Skincare & cosmetics | + +All subscriptions include free shipping. Active subscribers also receive 10% off +non-subscription purchases and earn double loyalty points on subscription deliveries. +Free shipping for subscribers applies regardless of order amount (unlike Standard +members who require a 20,000 won minimum). + +### Can I pause or skip a delivery? + +Yes. In "Settings > Subscriptions", tap "Pause" to skip the next delivery, or "Pause +Subscription" to halt all deliveries for up to 3 months. No charge during the pause +period. Resume anytime with no penalty. + +### How do I cancel a subscription? + +Go to "Settings > Subscriptions" and tap "Cancel". Cancellation takes effect after the +current billing cycle ends — no cancellation fee. Any remaining subscription credit is +refunded within 7 business days. You can resubscribe at any time, but previous pricing +is not guaranteed. + +### Can I customize what goes in my box? + +Partially. Each box has 3–5 "core items" selected by our curation team and 1–2 "choice +items" you can pick from a rotating catalog. Choice item selection opens 10 days before +each shipping date. If you do not select, our team picks based on your profile. + +### Is there a minimum commitment period? + +No. All subscriptions are month-to-month with no minimum commitment. Loyalty bonuses: +6 consecutive months earns 10,000 points; 12 consecutive months earns a 30,000 won coupon. diff --git a/src/main/resources/data/layer1_faq/support.md b/src/main/resources/data/layer1_faq/support.md new file mode 100644 index 0000000..28d6a1b --- /dev/null +++ b/src/main/resources/data/layer1_faq/support.md @@ -0,0 +1,31 @@ +# Customer Support + +## Cholog Corporation — Official FAQ + +### What support channels are available and when? + +| Channel | Hours | Notes | +|---------|-------|-------| +| Live Chat | 8 AM – 10 PM KST daily (incl. weekends) | Available on website and app. Avg. wait < 90 sec | +| Email | Business days, reply within 12 hours | support@cholog.kr | +| Phone | Mon–Fri 9 AM – 6 PM KST | 1588-0000 (general); 1588-0002 (VIP priority line) | +| KakaoTalk | 24/7 AI assistant | Search @초록. Human handoff during Live Chat hours | + +### Is customer support available in English? + +Yes. Live Chat and email support are available in Korean and English. Phone support is +primarily Korean, but English-speaking agents can be connected upon request. Response +times for English inquiries may be slightly longer during peak hours. + +### Can I schedule a phone callback? + +Yes. In the Help Center, select "Request Callback". Enter your phone number and preferred +time slot. Callbacks are available Monday to Friday, 9 AM to 6 PM KST. If you are a VIP +member, you can request priority callbacks via the dedicated line at 1588-0002. + +### How do I escalate an unresolved issue? + +If your issue is not resolved within 48 hours, reply to the support ticket email with +"ESCALATE" in the subject line. This routes your case to a senior support manager who +responds within 4 hours. VIP members can also request escalation via the dedicated +phone line at 1588-0002. diff --git a/src/main/resources/data/layer2_policies/current/complaint-handling.md b/src/main/resources/data/layer2_policies/current/complaint-handling.md new file mode 100644 index 0000000..116cf5d --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/complaint-handling.md @@ -0,0 +1,45 @@ +--- +title: Customer Complaint Handling Process +version: v2 +status: current +effective_date: 2024-02-01 +department: customer_service +supersedes: complaint-handling-v1.md +category: support +--- + +# Customer Complaint Handling Process v2 + +## Response SLA + +| Severity | Acknowledgment | Full Resolution | +|----------|---------------|-----------------| +| Critical (safety, fraud) | 30 minutes | 4 hours | +| High (wrong item, significant damage) | 2 hours | 24 hours | +| Standard (quality, delay) | 2 hours | 48 hours | +| Low (suggestion, general feedback) | 2 hours | 5 business days | + +## Escalation Path + +1. **Tier 1**: Front-line CS agent (Live Chat, Phone, Email) +2. **Tier 2**: Senior CS specialist — triggered by: customer request ("ESCALATE"), 48-hour + unresolved, or agent judgment for complex cases +3. **Tier 3**: CS Manager — triggered by: Tier 2 unresolved after 4 hours, VIP member + request, media/legal threat +4. **Legal / Regulatory**: For PIPA complaints, consumer protection agency referrals + +## Compensation Guidelines + +| Issue | Compensation | +|-------|-------------| +| Wrong item shipped | Free return + re-ship + 5,000 won coupon | +| Defective item | Free return + full refund or replacement | +| Delivery delay > 3 business days (non-island) | 2,000 won coupon | +| System error causing overcharge | Full refund + 1,000 won coupon | +| Confirmed counterfeit product | Full refund + 10,000 won coupon | + +## VIP Member Handling + +VIP members are routed directly to Tier 2 on first contact. If contact is via the dedicated +line (1588-0002), response time targets are halved. VIP complaints are flagged in the CRM +for priority tracking. diff --git a/src/main/resources/data/layer2_policies/current/marketplace-seller-agreement.md b/src/main/resources/data/layer2_policies/current/marketplace-seller-agreement.md new file mode 100644 index 0000000..622933b --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/marketplace-seller-agreement.md @@ -0,0 +1,53 @@ +--- +title: Marketplace Seller Agreement +version: v2 +status: current +effective_date: 2024-03-01 +department: marketplace +supersedes: marketplace-seller-agreement-v1.md +category: marketplace +--- + +# Marketplace Seller Agreement v2 + +## Eligibility + +Sellers must provide: valid Korean business registration certificate (사업자등록증), +product catalog with at least 5 SKUs, and a Korean bank account for settlement. +Applications are reviewed within 5 business days. Approval is at Cholog's sole discretion. + +## Commission Structure + +| Product Category | Commission Rate | +|-----------------|----------------| +| Electronics | 8% | +| Fashion & Apparel | 12% | +| Food & Beverage | 10% | +| Home & Living | 8% | +| Cosmetics | 10% | +| Books & Media | 5% | +| General | 10% | + +Commission is calculated on the sale price including VAT, deducted from the settlement amount. + +## Seller Performance Standards + +Sellers must maintain: +- Customer satisfaction rating ≥ **4.2 / 5.0** +- Order cancellation rate ≤ 3% +- Dispatch-on-time rate ≥ 95% +- Return dispute resolution within **3 business days** + +Failure to meet standards for 2 consecutive months results in a suspension notice. +Three suspension notices within 12 months result in permanent delisting. + +## Return Policy (Seller Obligation) + +Each seller sets their own return policy (minimum 3 days, maximum 30 days from delivery). +Return policy must be clearly stated on each product page. Cholog buyer protection overrides +seller policy in cases of item not received or item significantly not as described. + +## Settlement + +Settlements are processed bi-weekly (1st and 15th of each month) for all orders where +the return window has closed. Fees and commissions are itemized in the seller dashboard. diff --git a/src/main/resources/data/layer2_policies/current/membership-tiers.md b/src/main/resources/data/layer2_policies/current/membership-tiers.md new file mode 100644 index 0000000..9bf7edc --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/membership-tiers.md @@ -0,0 +1,46 @@ +--- +title: Membership Tier Policy +version: v3 +status: current +effective_date: 2024-01-01 +department: marketing +supersedes: membership-tiers-v2.md +category: membership +--- + +# Membership Tier Policy v3 + +## Tier Qualification + +| Tier | Annual Spending Threshold | Evaluation Date | +|------|--------------------------|-----------------| +| Standard | Default (no minimum) | — | +| Plus | ≥ 200,000 won | January 1st each year | +| VIP | ≥ 800,000 won | January 1st each year | + +Annual spending is calculated from January 1 to December 31. Returned orders are deducted +from the annual spending total. Tier downgrades take effect on the 1st of the following month +after a returned item pushes the total below the threshold. + +## Benefits by Tier + +| Benefit | Standard | Plus | VIP | +|---------|---------|------|-----| +| Point earning rate | 1% | 3% | 5% | +| Free shipping | Economy ≥ 20,000 won | Economy on all orders | Priority on all orders | +| Birthday bonus | 20% off coupon | 20% off + gift | 20% off + gift | +| Early product access | — | — | Yes | +| Dedicated support line | — | — | 1588-0002 | +| Point expiry | 12 months | 12 months | 12 months | + +## Point Earning + +Points are calculated on the pre-shipping, pre-discount order total. Points are credited +**7 days after delivery confirmation** and expire **12 months** from the date earned. +Minimum redemption is 1,000 points. 1 point = 1 won. + +## Tier Status Review + +Tier status is reviewed annually on January 1st. Members who do not meet the threshold in +the prior year are downgraded at the start of the new year. There is no partial-year +upgrade grace period. diff --git a/src/main/resources/data/layer2_policies/current/point-earning-rules.md b/src/main/resources/data/layer2_policies/current/point-earning-rules.md new file mode 100644 index 0000000..9d95ccb --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/point-earning-rules.md @@ -0,0 +1,50 @@ +--- +title: Loyalty Point Earning & Redemption Rules +version: v3 +status: current +effective_date: 2024-01-01 +department: marketing +supersedes: point-earning-rules-v2.md +category: loyalty +--- + +# Loyalty Point Earning & Redemption Rules v3 + +## Earning Rates + +| Membership Tier | Base Earning Rate | +|----------------|------------------| +| Standard | 1% of order subtotal | +| Plus | 3% of order subtotal | +| VIP | 5% of order subtotal | + +Order subtotal = item price × quantity (before shipping and discount). + +## Bonus Point Events + +| Event | Bonus | +|-------|-------| +| Green Packaging | 200 points per order | +| Photo/video review | 500 points | +| Electronics take-back | 2,000 points per item | +| Daily check-in (Day 7) | 500 points or random coupon | +| 30-day consecutive check-in | 10,000 won coupon | +| Active subscriber purchases | 2× base rate on subscription deliveries | + +## Redemption Rules + +- Minimum redemption: **1,000 points per order** +- 1 point = 1 won +- Points can be used on any purchase, combined with one coupon +- Points cannot be redeemed for cash or transferred between accounts + +## Point Expiry + +All points expire **12 months from the date earned**. There is no extension or restoration +of expired points. Points approaching expiry (within 30 days) trigger an app push notification. + +## Point Deduction on Returns + +Points earned from a returned order are deducted within 24 hours of refund processing. +If the resulting balance is negative, the deficit is carried forward and deducted from +future earned points. diff --git a/src/main/resources/data/layer2_policies/current/return-policy-v3.md b/src/main/resources/data/layer2_policies/current/return-policy-v3.md new file mode 100644 index 0000000..ec9faea --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/return-policy-v3.md @@ -0,0 +1,60 @@ +--- +title: Return & Refund Policy +version: v3 +status: current +effective_date: 2024-04-01 +department: customer_service +supersedes: return-policy-v2.md +category: returns +--- + +# Return & Refund Policy v3 + +## Standard Return Window + +All products purchased directly from Cholog Corporation may be returned within **14 calendar +days** of delivery, provided the item is unopened or, if opened, unused and in original +packaging with all accessories and documentation. + +### Exceptions by Product Category + +| Category | Return Window | Condition | +|----------|--------------|-----------| +| Electronics | 14 days | Sealed or with all accessories intact | +| Food & Beverages | Not returnable | Perishable goods | +| Cosmetics & Skincare | 14 days | Unused, sealed | +| Underwear / Swimwear | Not returnable | Hygiene reasons | +| Subscription Box | 14 days (individual items) | Per standard policy | +| Clearance Items | Not returnable | Final sale | +| Digital Downloads | Not returnable | Final sale | +| Personalized Items | Not returnable | Custom-made | +| Marketplace Items | Per seller policy | See seller page | + +## Handling Fee for Non-Defect Returns + +If the return reason is change of mind, wrong size, or any non-defect reason, a handling fee +of **3,000 won** is deducted from the refund. Defect-related returns incur no fee. + +## Refund Timeline + +| Payment Method | Refund Time | +|----------------|-------------| +| Credit card (domestic) | 3–5 business days after inspection | +| Kakao Pay / Naver Pay | Typically instant after inspection | +| Bank transfer | Up to 7 business days | +| International card | 7–14 business days | +| Cholog Pay | Instant | + +## Marketplace Items + +Items sold by third-party Marketplace sellers follow the seller's own return policy, not +Cholog Corporation's standard policy. Return windows vary from 3 to 30 days depending on +the seller. Cholog's buyer protection guarantee covers items not received or significantly +not as described, regardless of the seller's policy. + +## Exchange Policy + +Customers may request an exchange instead of a refund. Select "Exchange" when initiating +the return process. Exchanges are for the same product (different size or color only). +Price differences are charged or refunded accordingly. Exchanges are processed within +3–5 business days after the original item is collected. diff --git a/src/main/resources/data/layer2_policies/current/shipping-cold-chain.md b/src/main/resources/data/layer2_policies/current/shipping-cold-chain.md new file mode 100644 index 0000000..96f88b8 --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/shipping-cold-chain.md @@ -0,0 +1,52 @@ +--- +title: Cold-Chain Shipping Policy (Fresh & Frozen) +version: v2 +status: current +effective_date: 2024-01-01 +department: logistics +category: shipping +--- + +# Cold-Chain Shipping Policy v2 + +## Scope + +This policy applies to all products categorized as "Fresh", "Chilled", or "Frozen" on the +Cholog platform, including fresh produce, dairy, meat, seafood, and temperature-sensitive +supplements. + +## Courier & Packaging + +All cold-chain orders are shipped exclusively via **CJ Logistics Fresh**. Orders are packed +in insulated polystyrene boxes with gel ice packs rated for 24–36 hours of temperature +control. Frozen items are packed with dry ice where required. + +## Delivery Schedule + +Cold-chain delivery is available **Monday through Thursday only**. This restriction prevents +packages from sitting in transit over a weekend, which would exceed the ice pack rating. + +- Orders placed by 11 AM KST Monday–Thursday dispatch same day +- Orders placed Thursday after 11 AM dispatch the following Monday +- No cold-chain delivery on Friday, Saturday, Sunday, or public holidays + +## Fees + +| Order Amount | Cold-Chain Fee | +|-------------|----------------| +| Any amount | Flat 4,000 won | + +Cold-chain fee applies regardless of membership tier or free-shipping eligibility. +Cold-chain orders cannot be combined with standard-tier shipments. + +## Delivery Area + +Cold-chain delivery is available to all mainland Korean addresses and Jeju Island (add 2 +business days). Delivery to remote offshore islands (Ulleungdo, Dokdo, etc.) is not +available for cold-chain orders due to transit time constraints. + +## Returns + +Fresh and frozen products are non-returnable due to food safety regulations, except in cases +of: item received damaged, wrong item shipped, or item arrived visibly spoiled. In such cases, +a photo must be submitted within 24 hours of delivery. diff --git a/src/main/resources/data/layer2_policies/current/shipping-standard.md b/src/main/resources/data/layer2_policies/current/shipping-standard.md new file mode 100644 index 0000000..183cabb --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/shipping-standard.md @@ -0,0 +1,51 @@ +--- +title: Standard Shipping Policy +version: v4 +status: current +effective_date: 2024-01-01 +department: logistics +supersedes: shipping-standard-2023.md +category: shipping +--- + +# Standard Shipping Policy v4 + +## Shipping Tiers + +| Tier | Fee | Delivery Time | Notes | +|------|-----|---------------|-------| +| Economy | 2,500 won | 2–4 business days | Free for orders ≥ 20,000 won (Standard members) | +| Priority | 5,000 won | Next business day by noon | Free for Gold / VIP members | +| Same-Day | 8,000 won | Same day | Seoul, Incheon, Gyeonggi only; order before 10 AM KST | + +## Free Shipping Thresholds + +- **Standard members**: Economy shipping free on orders ≥ **20,000 won** +- **Plus members**: Economy shipping free on all orders +- **VIP members**: Priority shipping free on all orders +- **Active subscribers**: Priority shipping free on all orders (including non-subscription purchases) + +## Dispatch Cutoff + +Orders confirmed before **1 PM KST** on business days are dispatched the same afternoon. +Orders after 1 PM or on weekends/public holidays are queued for the next business day. + +## Courier Partners + +| Type | Courier | +|------|---------| +| Standard | Hanjin Express | +| Cold-chain | CJ Logistics Fresh | +| Large item | Lotte Global Logistics | + +## Jeju and Offshore Islands + +Delivery to Jeju, Ulleungdo, Geojedo, and all major offshore islands carries no surcharge +but takes an additional 1–2 business days. Same-day delivery is not available for island +addresses. + +## Failed Delivery + +The courier makes two delivery attempts. After both fail, the package is held at the nearest +Hanjin depot for 7 calendar days. An SMS is sent with pickup instructions. If uncollected, +the item is returned and a full refund is issued minus the original shipping fee. diff --git a/src/main/resources/data/layer2_policies/current/subscription-terms.md b/src/main/resources/data/layer2_policies/current/subscription-terms.md new file mode 100644 index 0000000..cd693ea --- /dev/null +++ b/src/main/resources/data/layer2_policies/current/subscription-terms.md @@ -0,0 +1,50 @@ +--- +title: Subscription Service Terms & Conditions +version: v2 +status: current +effective_date: 2024-01-01 +department: marketing +category: subscription +--- + +# Subscription Service Terms & Conditions v2 + +## Available Plans + +| Plan | Price | Billing Cycle | Shipping | +|------|-------|---------------|----------| +| Essentials Box | 29,900 won | Monthly | Free | +| Coffee Club | 19,900 won | Bi-weekly | Free | +| Beauty Box | 39,900 won | Monthly | Free | + +## Subscriber Benefits + +Active subscribers (any plan) receive: +- Free shipping on **all** orders (including non-subscription purchases), regardless of + order amount — this supersedes the Standard member 20,000 won free-shipping threshold +- 10% discount on all non-subscription purchases +- Double loyalty points on subscription deliveries +- Early access to new product launches (equivalent to VIP benefit) + +## Billing + +Automatic charges occur on the **first business day of each billing cycle**. A reminder +notification is sent 3 days before each charge. If the charge fails, we retry once after +24 hours. After two failed attempts, the subscription is paused and an email notification +is sent. + +## Pause & Cancellation + +- **Skip next delivery**: Available up to 48 hours before the scheduled dispatch date +- **Pause (up to 3 months)**: No charge during pause; resume anytime +- **Cancel**: Effective after the current billing cycle. No cancellation fee. + Remaining prepaid subscription credit refunded within 7 business days. + +## Loyalty Bonuses + +| Consecutive Months | Bonus | +|-------------------|-------| +| 6 months | 10,000 loyalty points | +| 12 months | 30,000 won coupon | + +Consecutive month count resets to zero if the subscription is canceled (not paused). diff --git a/src/main/resources/data/layer2_policies/internal/cs-team-return-exceptions.md b/src/main/resources/data/layer2_policies/internal/cs-team-return-exceptions.md new file mode 100644 index 0000000..62ea415 --- /dev/null +++ b/src/main/resources/data/layer2_policies/internal/cs-team-return-exceptions.md @@ -0,0 +1,44 @@ +--- +title: CS Team — Return Exception Guidelines (Internal) +version: v4 +status: internal +effective_date: 2024-04-01 +department: customer_service +audience: cs_agents_only +category: returns +--- + +# CS Team Return Exception Guidelines (INTERNAL — Do Not Share with Customers) + +This document supplements the official Return & Refund Policy v3. These exceptions are +applied at CS agent discretion and are not publicly advertised. + +## VIP Member Extended Return Window + +VIP members (spend ≥ 800,000 won/year) may be granted a **30-day return window** at agent +discretion for the following cases: +- Item was a gift (recipient needs time to evaluate) +- Customer had a documented medical or travel absence during the standard 14-day window +- Item was defective but defect only became apparent after extended use (e.g., battery) + +**Process**: Agent must log the exception in CRM with reason code "VIP-EXTENDED" and attach +supervisor approval (Tier 2 or above). No exception for Marketplace items. + +## Loyalty Point Retention on High-Value Returns + +For returns of orders > 100,000 won by Plus or VIP members, agents may offer to retain +50% of the earned points as goodwill. Apply only when customer explicitly complains about +point loss. Log with reason code "POINT-GOODWILL". + +## Handling Fee Waiver + +The 3,000 won non-defect handling fee may be waived for: +- First-time return by the customer (check CRM return history) +- Returns where the delivery was delayed by > 2 business days (check dispatch log) +- VIP members (always waive, no approval required) + +## Subscription Box Individual Item Returns + +Although the official policy states individual items in a subscription box can be returned +within 14 days, agents may approve returns for up to 30 days for active long-term +subscribers (6+ consecutive months) at Tier 2 discretion. diff --git a/src/main/resources/data/layer2_policies/internal/holiday-shipping-memo-2024.md b/src/main/resources/data/layer2_policies/internal/holiday-shipping-memo-2024.md new file mode 100644 index 0000000..d89a660 --- /dev/null +++ b/src/main/resources/data/layer2_policies/internal/holiday-shipping-memo-2024.md @@ -0,0 +1,38 @@ +--- +title: Holiday Season Shipping Schedule — 2024 +version: v1 +status: internal +effective_date: 2024-01-01 +department: logistics +audience: cs_agents +category: shipping +--- + +# Holiday Season Shipping Schedule 2024 (INTERNAL MEMO) + +## Lunar New Year (설날) — February 9–12, 2024 + +**Dispatch suspension**: February 7 (Wed) 1 PM KST through February 13 (Tue) 9 AM KST. +- Orders placed February 7 after 1 PM through February 12 will dispatch February 13. +- Priority shipping SLA (next business day by noon) does NOT apply during this period. +- Inform customers: deliveries during this period may be delayed by 3–5 additional business days. +- Cold-chain orders: suspended February 5 (Mon) — next available dispatch February 13. + +## Chuseok — September 16–18, 2024 + +**Dispatch suspension**: September 13 (Fri) 1 PM KST through September 19 (Thu) 9 AM KST. +- Orders placed September 13 after 1 PM through September 18 will dispatch September 19. +- Expect high volume surge before/after: delivery delays of 1–2 extra days are normal. +- Cold-chain orders: suspended September 11–19. + +## Year-End / New Year — December 31, 2024 – January 2, 2025 + +- December 31 dispatch cutoff: 11 AM KST (earlier than usual). +- January 1 (New Year's Day): No dispatch. +- Normal operations resume January 2. + +## Agent Instructions + +When customers ask about delivery during holiday periods, quote the extended estimates above. +Do NOT promise next-business-day delivery for Priority orders during suspension windows. +Proactively inform customers who placed orders close to holiday cutoffs via outbound KakaoTalk. diff --git a/src/main/resources/data/layer2_policies/internal/new-agent-onboarding.md b/src/main/resources/data/layer2_policies/internal/new-agent-onboarding.md new file mode 100644 index 0000000..a826a5b --- /dev/null +++ b/src/main/resources/data/layer2_policies/internal/new-agent-onboarding.md @@ -0,0 +1,50 @@ +--- +title: New CS Agent Onboarding Guide +version: v3 +status: internal +effective_date: 2024-01-01 +department: customer_service +audience: new_cs_agents +category: training +--- + +# New CS Agent Onboarding Guide (INTERNAL) + +Welcome to the Cholog CS team. This guide summarizes the most common customer scenarios +and the correct responses. Always refer to the official policy documents for authoritative +answers. When in doubt, escalate to Tier 2. + +## Top 5 Most Common Inquiries + +1. **배송 조회** — Check order in internal dashboard. Give Hanjin tracking number. + If status is "Preparing" for > 2 business days, flag as delayed dispatch to logistics. + +2. **반품 신청** — Confirm within return window (14 days from delivery per v3 policy). + Check item category — some are non-returnable (clearance, cosmetics, digital, personalized). + Remind customer: non-defect returns incur 3,000 won handling fee. + +3. **환불 문의** — Confirm inspection is complete. Quote timeline by payment method: + 카드 3–5일, 카카오/네이버페이 즉시, 계좌이체 최대 7일. + +4. **포인트 문의** — 1점=1원, 최소 1,000점 사용, 12개월 유효기간. + Tier별 적립률: Standard 1%, Plus 3%, VIP 5%. (구버전 혼동 주의: 예전엔 3%/5%/7%였음) + +5. **멤버십 등급** — Plus = 연 20만원 이상, VIP = 연 80만원 이상. + 등급 평가일: 매년 1월 1일. (2023년 기준과 다름: 구버전은 Plus=15만, VIP=60만이었음) + +## Important Gotchas for New Agents + +- **마켓플레이스 상품**: Cholog 기본 반품 정책 미적용. 판매자 정책 확인 필수. +- **구독 회원 배송비**: 2만원 미만 주문도 무료 배송 (구독 혜택). Standard와 다름. +- **VIP 반품 예외**: cs-team-return-exceptions.md 참고 — 공개 정책과 다른 내부 기준 있음. +- **냉장 배송 요일**: 월~목만 가능. 금요일 주문하면 다음 월요일 발송. +- **세금계산서**: 당월 내에만 발급 신청 가능. 전월 주문 건 불가. + +## Escalation Rules + +Escalate to Tier 2 if: +- Customer is VIP and unhappy after your first response +- Return/refund claim exceeds 200,000 won +- Customer mentions legal action, press, or social media +- Fraud or counterfeit suspected +- Issue involves cold-chain spoilage diff --git a/src/main/resources/templates/chat.html b/src/main/resources/templates/chat.html new file mode 100644 index 0000000..1534990 --- /dev/null +++ b/src/main/resources/templates/chat.html @@ -0,0 +1,623 @@ + + + + + + 초록 고객지원 챗봇 + + + +
+ + +
+
+
+

상담

+
응답 대기 중
+
+ +
+ +
+
+
+

+ 안녕하세요. 초록 고객지원 챗봇입니다. +

+
+ 지금 +
+
+ +
+
+ +
+
+ +
+ + +
+
+
+ + + +