[우분투] terminal에서 복사(copy)와 붙여넣기(paste)

참고 문서: 

• https://crfm.stanford.edu/2023/03/13/alpaca.html

• https://github.com/tatsu-lab/stanford_alpaca/tree/65512697dc67779a6e53c267488aba0ec4d7c02a 

Stanford에서 만든 Alpaca model 

• Meta의 LLaMA 7B model 사용 
• "Alpaca is a language model fine-tuned using supervised learning from a LLaMA 7B model on 52K instruction-following demonstrations generated from OpenAI’s text-davinci-003" 
• 출처: https://crfm.stanford.edu/2023/03/13/alpaca.html 

Alpaca 용 학습 데이터 구축소규모 학습 데이터를 GPT를 이용해서 대량으로 자동 생성함 

• 수동으로 소규모 데이터를 구축하고, 이 데이터를 seed로 해서 GPT를 이용해서 52K개의 데이터를 자동 생성
• 175개 데이터를 수동으로 구축한다. 

• 수동 데이터 예시 (tatsulab/stanford github)

{"id": "seed_task_0", "name": "breakfast_suggestion", "instruction": "Is there anything I can eat for a breakfast that doesn't include eggs, yet includes protein, and has roughly 700-1000 calories?", "instances": [{"input": "", "output": "Yes, you can have 1 oatmeal banana protein shake and 4 strips of bacon. The oatmeal banana protein shake may contain 1/2 cup oatmeal, 60 grams whey protein powder, 1/2 medium banana, 1tbsp flaxseed oil and 1/2 cup watter, totalling about 550 calories. The 4 strips of bacon contains about 200 calories."}], "is_classification": false} 

{"id": "seed_task_1", "name": "antonym_relation", "instruction": "What is the relation between the given pairs?", "instances": [{"input": "Night : Day :: Right : Left", "output": "The relation between the given pairs is that they are opposites."}], "is_classification": false}

• GPT ( OpenAI' text-davinci-003 모델 )로 학습 데이터 생성하기 
• GPT 입력 데이터: 수동으로 만든 prompt 데이터와 175개의 수동 데이터로 생성 
• GPT 입력 데이터 예시 (코드 참조해서 만든 데이터) 
• 아래 예시에서 "List of 20 tasks"까지는 prompt 에 있는 내용
• "List of 20 tasks" 아래 내용은 수동으로 만든 175개 데이터 중에서 임의로 선정해서 생성 

You are asked to come up with a set of 20 diverse task instructions. These task instructions will be given to a GPT model and we will evaluate the GPT model for completing the instructions. Here are the requirements: 

1. Try not to repeat the verb for each instruction to maximize diversity. 

2. The language used for the instruction also should be diverse. For example, you should combine questions with imperative instrucitons. 

3. The type of instructions should be diverse. The list should include diverse types of tasks like open-ended generation, classification, editing, etc. 

4. A GPT language model should be able to complete the instruction. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action. 

5. The instructions should be in English. 

6. The instructions should be 1 to 2 sentences long. Either an imperative sentence or a question is permitted. 

7. You should generate an appropriate input to the instruction. The input field should contain a specific example provided for the instruction. It should involve realistic data and should not contain simple placeholders. The input should provide substantial content to make the instruction challenging but should ideally not exceed 100 words. 

8. Not all instructions require input. For example, when a instruction asks about some general information, "what is the highest peak in the world", it is not necssary to provide a specific context. In this case, we simply put "<noinput>" in the input field. 

9. The output should be an appropriate response to the instruction and the input. Make sure the output is less than 100 words. 

List of 20 tasks:

###

1. Instruction: To make the pairs have the same analogy, write the fourth word.

1. Input: Instability : Turmoil:: Change :

1. Output: Instability : Turmoil :: Change : Revolution

###

2. Instruction: Given a description of the symptom, identify the possible disease and suggest some medicine.

2. Input: I have a fever and I am coughing.

2. Output: The possible disease is pneumonia. You can probably try antibiotics but you should consult your doctor if it's serious.

출처: https://crfm.stanford.edu/2023/03/13/alpaca.html