Move clk100k_base tokenizer to docker image (#3411)

### What problem does this PR solve?

Move the tiktoken of cl100k_base into docker image

issue: #3338 

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
This commit is contained in:
Jin Hai
2024-11-15 10:18:40 +08:00
committed by GitHub
parent 220aaddc62
commit 996c94a8e7
4 changed files with 15 additions and 4 deletions

View File

@@ -119,6 +119,9 @@ COPY nltk_data /root/nltk_data
COPY tika-server-standard-3.0.0.jar tika-server-standard-3.0.0.jar.md5 ./
ENV TIKA_SERVER_JAR="file:///ragflow/tika-server-standard.jar"
# Copy cl100k_base
COPY 9b5ad71b2ce5302211f9c61530b329a4922fc6a4 ./
# Copy compiled web pages
COPY --from=builder /ragflow/web/dist /ragflow/web/dist