代码编织梦想

iclr 2023 | self-爱代码爱编程

大家好,我是HxShine。 今天分享一篇Google Research, Brain Team的一篇文章,SELF-CONSISTENCY IMPROVES CHAIN OF THOUGHT REASONING IN LANGUAGE MODELS[1]:利用自洽性提高语言模型中的思维链推理效果 这篇文章方法非常简单但是效果非常好,OpenAI的An

a comprehensive capability analysis of gpt-爱代码爱编程

一、概述 Motivation:GPT系列的模型,像GPT-3,CodeX,InstructGPT,ChatGPT,尽管很多人关注他们能力的不同,但是很少关注GPT系列模型随着时间变化其能力的变化情况。 Methods:

openai | let’s verify step by step详细解读_let's verify step by step-爱代码爱编程

一、概述 title:Let’s Verify Step by Step 论文地址:https://arxiv.org/abs/2305.20050 代码:GitHub - openai/prm800k: 800,000 step-level correctness labels on LLM solutions to MATH problems

acl2023 | 大模型如何快速构建指令遵循数据集?self-instruct:用175条种子数据追上instructgpt001效果-爱代码爱编程

一、概述 title:SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions 论文地址:https://arxiv.org/abs/2212.10560 代码:GitHub - yizhongw/self-instruct: Aligning pretraine

acl2023 | webcpm:清华发布中文lfqa 数据集,探索搜索引擎和plm大模型结合新范式-爱代码爱编程

一、概述 title:WEBCPM: Interactive Web Search for Chinese Long-form Question Answering 论文地址:https://arxiv.org/abs/2305.06849 代码:https://github.com/thunlp/WebCPM 1.1 Motivation 开发

google | cot(chain of thought)开山之作,利用思维链提升复杂问题推理能力-爱代码爱编程

一、概述 title:Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 论文地址:https://arxiv.org/abs/2201.11903 auto COT代码【COT升级版本】:GitHub - amazon-science/auto-cot: Off

qlora | 48g内存训练24小时,改进版4-bit量化技术微调650亿参数的模型达到chatgpt99.3%的效果-爱代码爱编程

一、概述 title:QLORA: Efficient Finetuning of Quantized LLMs 论文地址:https://arxiv.org/pdf/2305.14314.pdf 代码:GitHub - artidoro/qlora: QLoRA: Efficient Finetuning of Quantized LLMs and

acl2023 | 黑盒大模型如何微调?清华decoder tuning方法提升大模型few-shot场景效果-爱代码爱编程

一、概述 title:Decoder Tuning: Efficient Language Understanding as Decoding 论文地址:https://arxiv.org/abs/2212.08408 代码:GitHub - thunlp/DecT 二、Motivation 现在有很多模型只提供API,没法直接训练,并且是按请求

emlp2021 | google大模型微调经典论文prompt tuning-爱代码爱编程

一、概述 title:The Power of Scale for Parameter-Efficient Prompt Tuning 论文地址:https://arxiv.org/abs/2104.08691 代码:GitHub - google-research/prompt-tuning: Original Implementation of

acl2022 | 大模型微调哪家好?小孩子才做选择,成年人当然是全都要-爱代码爱编程

一、概述 title:UNIPELT: A Unified Framework for Parameter-Efficient Language Model Tuning 论文地址:https://arxiv.org/abs/2110.07577 代码:GitHub - morningmoni/UniPELT: Code for paper "Uni

微软 lora| 使用万分之一的参数微调你的gpt3模型-爱代码爱编程

一、概述 title:LORA: LOW-RANK ADAPTATION OF LARGE LAN- GUAGE MODELS 论文地址:https://arxiv.org/abs/2106.09685 代码:GitHub - microsoft/LoRA: Code for loralib, an implementation of "LoRA:

清华p-tuning | gpt也能做nlu?清华推出p-tuning方法解决gpt系列模型fine-tuning效果比bert差问题-爱代码爱编程

一、概述 title:GPT Understands, Too 论文地址:https://arxiv.org/abs/2103.10385 代码:https://github.com/THUDM/P-tuning 1.1 Motivation GPTs模型利用传统的fine-tuning技术在NLU任务上效果比较差,比同等量级的BERT效果要差。

微软 | 把local小模型当作大语言模型的插件?-爱代码爱编程

一、概述 title:Small Models are Valuable Plug-ins for Large Language Models 论文地址:https://arxiv.org/abs/2305.08848 代码:https://github.com/JetRunner/SuperICL 1.1 Motivation 大语言模型想GP

chatgpt如何引入领域知识?mit团队利用gpt4做数据增强来提升小模型在特定领域的效果-爱代码爱编程

一、概述 title:Dr. LLaMA: Improving Small Language Models in Domain-Specific QA via Generative Data Augmentation 论文地址:Paper page - Dr. LLaMA: Improving Small Language Models in Doma

acl 2022 | chatglm微调神器p-tuning v2论文学习-爱代码爱编程

一、概述 title:P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks 论文地址:https://arxiv.org/abs/2110.07602 代码:GitHub - THUDM/P-tuning-v2:

google|只有大模型才能理解你举的例子(in-context learning)是什么-爱代码爱编程

一、概述 title:LARGER LANGUAGE MODELS DO IN-CONTEXT LEARNING DIFFERENTLY 论文地址:https://arxiv.org/abs/2303.03846 参考:https://www.xiaohongshu.com/user/profile/5f01057f0000000001003c91/

chatgpt如何引入新的知识?我们来看下acl2023 预训练模型能否对新注入的知识进行推理这篇文章-爱代码爱编程

一、概述 title:Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge 论文地址:https://arxiv.org/abs/2305.01651 相关代码: EKP数据和代码:GitHub - yasumasaono

plato-2: towards building an open-domain chatbot via curriculum learning论文学习-爱代码爱编程

一、概述 Motivation:直接提升PLATO的size训练不work Methods: 通过curriculum learning技术来构建一个高质量的开放领域机器人第一阶段:coarse-gained generation model:再简单的one-to-one框架下学习粗力度的回复生成模型第二阶段:精调的模型来提高多样性和选择best

learning to memorize entailment and discourse relations for persona-consistent dialogues论文学习-爱代码爱编程

一、大纲 二、详细内容 abstract ■ 对话系统的engagement和consistency非常重要 ■ 现有方法 ● 复杂的网络结构->大量的标注语料 ● 忽视了篇章的连贯性(discour

信息抽取模型优缺点总结和优化点梳理_hxshine的博客-爱代码爱编程

一、信息抽取模型优缺点总结 参考: NLP系列之封闭域联合抽取:CasRel、TPLinker、PRGC、PURE、OneRel,实在是太卷了!:https://zhuanlan.zhihu.com/p/4980895