atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>SMULET&#39;s BLOG</title>
  
  
  <link href="http://simuleite.github.io/atom.xml" rel="self"/>
  
  <link href="http://simuleite.github.io/"/>
  <updated>2025-10-04T01:30:04.357Z</updated>
  <id>http://simuleite.github.io/</id>
  
  <author>
    <name>SIMULEITE</name>
    
  </author>
  
  <generator uri="https://hexo.io/">Hexo</generator>
  
  <entry>
    <title>MoE</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/LLM/MoE/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/LLM/MoE/</id>
    <published>2025-10-03T16:00:00.000Z</published>
    <updated>2025-10-04T01:30:04.357Z</updated>
    
    <content type="html"><![CDATA[<h1 id="什么是moe"><a class="markdownIt-Anchor" href="#什么是moe"></a> 什么是MoE</h1><p>Mixture of Experts<br />MoE使用Sparse，MLP替换FFN</p><h1 id="为什么moe"><a class="markdownIt-Anchor" href="#为什么moe"></a> 为什么MoE</h1><p>不可能三角：Performance、Cost、Size。</p><h2 id="成本-性能"><a class="markdownIt-Anchor" href="#成本-性能"></a> 成本-性能</h2><p>传统的Dense模型，想要Scale Up，提升Size，一定会带动Computation提升，从而导致Cost太大。<br />Moe将Performance、Size解耦，Scale Up提升Size，但是Computation不会变化。（只激活部分Expert）</p><h2 id="单一职责knowledge"><a class="markdownIt-Anchor" href="#单一职责knowledge"></a> 单一职责Knowledge</h2><p>传统模型，Knowledge存储在FFN；MoE将不同领域知识放在不同的Expert里面，有专业化区分，单一职责。<br />参数量更大，也使MoE模型可以存储更多Knowledge。</p><h2 id="sparse"><a class="markdownIt-Anchor" href="#sparse"></a> Sparse</h2><p>CNN本质就是引入Sparse。Sparse引领Machine Learning/Deep Learning发展。<br />Sparse做的事是捕捉信息的low-dimension pattern，提升learning efficiency。（这也是CNN性能好于MLP的原因）</p><h1 id="moe比dense好在哪"><a class="markdownIt-Anchor" href="#moe比dense好在哪"></a> MoE比Dense好在哪？</h1><ol><li>MoE在总参量一样的情况下，都比同样参数的Dense要好（激活参数、总参数都是）</li><li>MoE训练时间更短：更低的Training、Validation Loss</li></ol><h1 id="deepseek-moe"><a class="markdownIt-Anchor" href="#deepseek-moe"></a> DeepSeek MoE</h1><h2 id="fine-grain-expert"><a class="markdownIt-Anchor" href="#fine-grain-expert"></a> Fine-Grain Expert</h2><p>把比较大的Expert分成多个小Experts</p><h2 id="shared-expert"><a class="markdownIt-Anchor" href="#shared-expert"></a> Shared Expert</h2><p>所有输入都会经过Shared Expert，不依赖Router</p><h2 id="效果验证"><a class="markdownIt-Anchor" href="#效果验证"></a> 效果验证</h2><p>从8-32，有提升；但是从32-64，几乎没有；Experts有一个Sweet Point</p><p>而Shared Expert其实没有什么用，属于人的错误先验假设</p><h1 id="训练时token分类"><a class="markdownIt-Anchor" href="#训练时token分类"></a> 训练时Token分类</h1><h2 id="token-choice"><a class="markdownIt-Anchor" href="#token-choice"></a> Token Choice</h2><p>根据每个Token做Router选择专家；<br />存在LoadBalance问题，被集中分配到某些Expert身上。<br />也就是说，有些Expert没有被充分训练</p><h2 id="expert-choice"><a class="markdownIt-Anchor" href="#expert-choice"></a> Expert Choice</h2><p>反过来，从Expert的角度分配Token；<br />存在Token Dropping，有些Token没有被选择到；<br />在Inference时，因为AutoRegressive，无法看到后面的Token，这个方法会有问题。解决方法是增加一个MLP中间层。</p><p>目前，Token Choice表现还是好一点</p><h1 id="moe的困难"><a class="markdownIt-Anchor" href="#moe的困难"></a> MoE的困难</h1><h2 id="sparse训练困难training-instability"><a class="markdownIt-Anchor" href="#sparse训练困难training-instability"></a> Sparse训练困难：Training Instability</h2><p>训练不稳定，容易Loss崩溃</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;什么是moe&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#什么是moe&quot;&gt;&lt;/a&gt; 什么是MoE&lt;/h1&gt;
&lt;p&gt;Mixture of Experts&lt;br /&gt;
MoE使用Sparse，MLP替换FFN&lt;/p&gt;
&lt;h1 id=&quot;为什</summary>
      
    
    
    
    
    <category term="LLM" scheme="http://simuleite.github.io/tags/LLM/"/>
    
  </entry>
  
  <entry>
    <title>LangGraph</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/LangGraph/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/LangGraph/</id>
    <published>2025-08-05T16:00:00.000Z</published>
    <updated>2025-09-08T01:06:31.055Z</updated>
    
    <content type="html"><![CDATA[<figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">source venv/bin/activate</span><br><span class="line">pip install langgraph python-dotenv openai</span><br></pre></td></tr></table></figure><h1 id="langgraph-demo"><a class="markdownIt-Anchor" href="#langgraph-demo"></a> LangGraph Demo</h1><h2 id="10-state"><a class="markdownIt-Anchor" href="#10-state"></a> 1.0 State</h2><p>State就是AI流转的全局变量</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">InputState</span>(<span class="title class_ inherited__">TypedDict</span>):</span><br><span class="line">    question: <span class="built_in">str</span></span><br><span class="line">    llm_answer: <span class="type">Optional</span>[<span class="built_in">str</span>] <span class="comment"># None or str</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">OutputState</span>(<span class="title class_ inherited__">TypedDict</span>):</span><br><span class="line">    answer: <span class="built_in">str</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">OverallState</span>(InputState, OutputState):</span><br><span class="line">    <span class="keyword">pass</span></span><br></pre></td></tr></table></figure><h2 id="20-node"><a class="markdownIt-Anchor" href="#20-node"></a> 2.0 Node</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">llm_node</span>(<span class="params">state: InputeState</span>):</span><br><span class="line">    msg = [</span><br><span class="line">        (<span class="string">&quot;system&quot;</span>, readMd(<span class="string">&quot;system.md&quot;</span>)),</span><br><span class="line">        (<span class="string">&quot;human&quot;</span>, state[<span class="string">&quot;question&quot;</span>])</span><br><span class="line">    ]</span><br><span class="line"></span><br><span class="line">    llm = ChatOpenAI(model=<span class="string">&quot;gpt-4o&quot;</span>)</span><br><span class="line"></span><br><span class="line">    resp = llm.invoke(msg)</span><br><span class="line">    <span class="keyword">return</span> &#123;<span class="string">&quot;answer&quot;</span>: resp.content&#125;</span><br></pre></td></tr></table></figure><h2 id="30-graph-compile"><a class="markdownIt-Anchor" href="#30-graph-compile"></a> 3.0 Graph Compile</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">builder = StateGraph(OverallState, <span class="built_in">input</span>=InputState, output=OutputState)</span><br><span class="line"></span><br><span class="line">builder.add_node(<span class="string">&quot;llm&quot;</span>, llm_node)</span><br><span class="line">builder.add_edge(START, <span class="string">&quot;llm&quot;</span>)</span><br><span class="line">builder.add_edge(<span class="string">&quot;llm&quot;</span>, END)</span><br><span class="line"></span><br><span class="line">graph = builder.<span class="built_in">compile</span>()</span><br></pre></td></tr></table></figure><h2 id="draw-graph"><a class="markdownIt-Anchor" href="#draw-graph"></a> Draw Graph</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">display(Image(graph.get_graph(xray=<span class="literal">True</span>).draw_mermaid_png()))</span><br></pre></td></tr></table></figure><h2 id="messages-history"><a class="markdownIt-Anchor" href="#messages-history"></a> Messages History</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">State</span>(<span class="title class_ inherited__">TypedDict</span>):</span><br><span class="line">    msgs: Annotated[<span class="built_in">list</span>, operator.add]</span><br></pre></td></tr></table></figure><h1 id="messagegraph"><a class="markdownIt-Anchor" href="#messagegraph"></a> MessageGraph</h1><p>使用Reducer追加消息，但是可以对已有消息做更新、合并、删除操作（Context Engine）</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">MessageGraph</span>(<span class="title class_ inherited__">StateGraph</span>):</span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">__init__</span>(<span class="params">self</span>) -&gt; <span class="literal">None</span>:</span><br><span class="line">        <span class="built_in">super</span>().__init__(Annotated[<span class="built_in">list</span>[AnyMessage], add_message])</span><br></pre></td></tr></table></figure><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">builder = MessageGraph()</span><br><span class="line"><span class="comment"># ...</span></span><br><span class="line">graph = builder.<span class="built_in">compile</span>()</span><br><span class="line"></span><br><span class="line">msgs2 = [HumanMessage(content=<span class="string">&quot;xxx&quot;</span>, <span class="built_in">id</span>=msg1.<span class="built_in">id</span>)]</span><br><span class="line"><span class="comment"># ID相同，覆盖消息</span></span><br><span class="line">add_messages(msgs1, msgs2)</span><br></pre></td></tr></table></figure><h1 id="structured-output"><a class="markdownIt-Anchor" href="#structured-output"></a> Structured Output</h1><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">UserInfo</span>(<span class="title class_ inherited__">BaseModel</span>):</span><br><span class="line">    name: <span class="built_in">str</span> = Field(description=<span class="string">&quot;The name of the user&quot;</span>)</span><br><span class="line">    <span class="comment"># ...</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Runnable对象</span></span><br><span class="line">structured_llm = llm.with_structured_output(UserInfo)</span><br><span class="line"></span><br><span class="line"><span class="comment"># UserInfo对象</span></span><br><span class="line">resp = structured_llm.invoke(msg)</span><br></pre></td></tr></table></figure><h1 id="tool-calling-agent"><a class="markdownIt-Anchor" href="#tool-calling-agent"></a> Tool Calling Agent</h1><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="comment"># Tool注解会拿到函数名、函数入参与函数注释</span></span><br><span class="line"><span class="meta">@tool</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">your_tool</span>(<span class="params">args</span>):</span><br><span class="line">    <span class="string">&quot;&quot;&quot;Description&quot;&quot;&quot;</span></span><br><span class="line"></span><br><span class="line">tools = [your_tool]</span><br><span class="line">tool_node = ToolNode(tools)</span><br></pre></td></tr></table></figure><h1 id="react-agent"><a class="markdownIt-Anchor" href="#react-agent"></a> ReAct Agent</h1><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 默认使用Agent State</span></span><br><span class="line"><span class="comment"># 注意，这是一个已经compile的图</span></span><br><span class="line">graph = create_react_agent(llm, tools=tools)</span><br></pre></td></tr></table></figure><h2 id="react-graph"><a class="markdownIt-Anchor" href="#react-graph"></a> ReAct Graph</h2><p>ReAct = 有条件调用 + 调用必返回</p><figure class="highlight python"><table><tr><td class="code"><pre><span class="line">workflow = StateGraph(State)</span><br><span class="line"></span><br><span class="line">workflow.add_node(<span class="string">&quot;agent&quot;</span>, call_model)</span><br><span class="line">workflow.add_node(<span class="string">&quot;tools&quot;</span>, tool_model)</span><br><span class="line"></span><br><span class="line">workflow.add_edge(START, <span class="string">&quot;agent&quot;</span>)</span><br><span class="line"></span><br><span class="line">workflow.add_conditional_edges(</span><br><span class="line">    <span class="string">&quot;agent&quot;</span>,</span><br><span class="line">    should_continue,</span><br><span class="line">    [<span class="string">&quot;tools&quot;</span>, END],</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 双向连接</span></span><br><span class="line">workflow.add_edge(<span class="string">&quot;tools&quot;</span>, <span class="string">&quot;agent&quot;</span>)</span><br><span class="line"></span><br><span class="line">app = workflow.<span class="built_in">compile</span>()</span><br></pre></td></tr></table></figure><h2 id="should_continue"><a class="markdownIt-Anchor" href="#should_continue"></a> should_continue</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">should_continue</span>(<span class="params">state: State</span>):</span><br><span class="line">    messages = state[<span class="string">&quot;message&quot;</span>]</span><br><span class="line">    last_mesaage = message[-<span class="number">1</span>]</span><br><span class="line">    <span class="keyword">if</span> <span class="keyword">not</span> last_message.tool_calls:</span><br><span class="line">        <span class="keyword">return</span> END</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">return</span> <span class="string">&quot;tools&quot;</span></span><br></pre></td></tr></table></figure><h2 id="call_model"><a class="markdownIt-Anchor" href="#call_model"></a> call_model</h2><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">async</span> <span class="keyword">def</span> <span class="title function_">call_model</span>(<span class="params">state: State, config: RunnableConfig</span>):</span><br><span class="line">    msgs = state[<span class="string">&quot;messages&quot;</span>]</span><br><span class="line">    resp = <span class="keyword">await</span> model.invoke(msgs, config)</span><br><span class="line">        <span class="keyword">return</span> &#123;<span class="string">&quot;messages&quot;</span>: resp&#125;</span><br></pre></td></tr></table></figure><h1 id="stream-output"><a class="markdownIt-Anchor" href="#stream-output"></a> Stream Output</h1><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">print_stream</span>(<span class="params">stream</span>):</span><br><span class="line">    <span class="keyword">for</span> sub_stream <span class="keyword">in</span> stream:</span><br><span class="line">        <span class="built_in">print</span>(sub_stream)</span><br></pre></td></tr></table></figure><h1 id="mcp"><a class="markdownIt-Anchor" href="#mcp"></a> MCP</h1><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">    <span class="attr">&quot;mcpServers&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;tool_name&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">            <span class="attr">&quot;command&quot;</span><span class="punctuation">:</span> <span class="string">&quot;npx&quot;</span><span class="punctuation">,</span></span><br><span class="line">            <span class="attr">&quot;args&quot;</span><span class="punctuation">:</span> <span class="punctuation">[</span><span class="string">&quot;your_arg&quot;</span><span class="punctuation">]</span><span class="punctuation">,</span></span><br><span class="line">            <span class="attr">&quot;transport&quot;</span><span class="punctuation">:</span> <span class="string">&quot;stdio&quot;</span></span><br><span class="line">        <span class="punctuation">&#125;</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br></pre></td></tr></table></figure><figure class="highlight python"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Configuration</span>()</span><br><span class="line"><span class="meta">    @staticmethod</span></span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">load_servers</span>(<span class="params">file_path: <span class="built_in">str</span> = <span class="string">&quot;servers_config.json&quot;</span></span>) -&gt; <span class="type">Dict</span>[<span class="built_in">str</span>, <span class="type">Any</span>]:</span><br><span class="line">        <span class="keyword">with</span> <span class="built_in">open</span>(file_path, <span class="string">&quot;r&quot;</span>, encoding=<span class="string">&quot;utf-8&quot;</span>) <span class="keyword">as</span> f:</span><br><span class="line">            <span class="keyword">return</span> json.load(f).get(<span class="string">&quot;mcpServers&quot;</span>, &#123;&#125;)</span><br><span class="line"></span><br><span class="line">cfg = Configuration()</span><br><span class="line">servers_cfg = Configuration.load_servers()</span><br><span class="line">mcp_client = MultiServerMCPClient(servers_cfg)</span><br><span class="line">tools = <span class="keyword">await</span> mcp_client.get_tools()</span><br><span class="line"></span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;figure class=&quot;highlight shell&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;source venv/bin/activate&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;pi</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>Multi-Agents</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Multi-Agents/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Multi-Agents/</id>
    <published>2025-06-26T16:00:00.000Z</published>
    <updated>2025-06-28T02:02:41.763Z</updated>
    
    <content type="html"><![CDATA[<h1 id="dont-build-multi-agents"><a class="markdownIt-Anchor" href="#dont-build-multi-agents"></a> Don’t Build Multi-Agents</h1><p><a href="https://cognition.ai/blog/dont-build-multi-agents">Cognition</a><br />构建长期运行的AI智能体系统，需要解决“可靠性”问题：</p><ol><li>上下文丢失、过长</li><li>状态混乱</li><li>错误累积</li></ol><p>例如，Multi-Agent思路需要构建规划Agent、解释Agent、执行Agent、SOP Agent。<br />然而，如果仅仅使用两个独立Agent，其生成结果会更加独立、隔绝，而不是相关联。<br />整体大于局部。局部的完整性不能保证整体的一致性。</p><h2 id="shared-data"><a class="markdownIt-Anchor" href="#shared-data"></a> Shared Data</h2><blockquote><p>Principle 1: Share context, and share full agent traces, not just individual messages.</p></blockquote><p>我们希望通过共享上下文解决一致性问题，但是不行。<br />Agent1和Agent2都是基于自己对Shared Data的理解工作，而不知道对方在做什么。<br />因此我们需要共享Traces，让一个Agent（例如解释Agent）对另一个Agent（例如执行Agent）进行Revision校正。<br />可是，只有垂直矫正，水平的Agent（多个执行Agents）之间仍然不知道对方在做什么。</p><h2 id="actions-mean-desisons"><a class="markdownIt-Anchor" href="#actions-mean-desisons"></a> Actions mean desisons</h2><blockquote><p>Principle 2: Actions carry implicit decisions, and conflicting decisions carry bad results.</p></blockquote><p>每个Agent的行为都必须基于同样的预期结果，而不能基于不清楚、有歧义的预期结果；否则整体很难保持风格统一。</p><h2 id="single-threaded-linear-agent"><a class="markdownIt-Anchor" href="#single-threaded-linear-agent"></a> Single-threaded Linear Agent</h2><p>鉴于上面两条，作者选择使用单Agent线性解决问题。<br />然而，这样做容易产生<code>context windows overflow</code>上下文溢出（因为线性Agent其实就是不断附带上一次的上下文进行下一次chat）。<br />我们引入总结压缩LLM解决上下文问题。</p><h2 id="claude-code设计模式"><a class="markdownIt-Anchor" href="#claude-code设计模式"></a> Claude Code设计模式</h2><p>Calude Code的智能体有两个特点：</p><ol><li>主Agent与子Agent不会并行运行</li><li>子Agent只回答简单问题，而不会编写代码<br />这样做有几个优点</li></ol><ul><li>避免上下文冲突：子Agent不包括主Agent的上下文，只回答清晰、具体的问题。</li><li>节省上下文：子Agent的操作也不保存在主Agent的上下文中。他们是解耦合的。</li></ul><h1 id="how-we-built-our-multi-agent-research-system"><a class="markdownIt-Anchor" href="#how-we-built-our-multi-agent-research-system"></a> How we built our multi-agent research system</h1><p><a href="https://www.anthropic.com/engineering/built-multi-agent-research-system">How we built our multi-agent research system</a><br />三种AI模式：</p><ol><li>Chat AI</li><li>Single Agent</li><li>Multiple Agents<br />Multi-Agents的优势在于回答开放、不确定的问题。传统的单Agent不适合研究，而多Agent并行搜索，最终总结出来的信息压缩性更强。</li></ol><blockquote><p>The essence of search is compression.</p></blockquote><p>Anthropic团队区分了两种模式：</p><ol><li>垂直模式：容易并行处理的任务，Leader Agent与多个Sub Agent交互</li><li>水平模式：不容易并行的任务、需要上下文共享的任务、Agent依赖强的任务，如编程，Leader Agent一步一步执行Sub stage</li></ol><p>Agent模式的token使用量是Chat模式的4倍；而Multi-Agent则是Chat模式的15倍。<br />Multi-Agent让token用量增加，因此更可能解决问题。同时也带来的高成本。</p><h2 id="prompt-engineering"><a class="markdownIt-Anchor" href="#prompt-engineering"></a> Prompt Engineering</h2><ol><li>Think like your agents.</li><li>Teach the orchestartor how to delegate.<br />例如，子问题如何划分？怎么确定它就是任务的最小可执行单元？<br />可以使用 明确预期结果-example输出格式-可用资源tools-任务边界不要做什么 这一套指令。</li><li>Scale effort to query complexity.<br />为prompt嵌入scaling rules，明确指出简单-中等-复杂任务分别分配多少subagents。这一条主要是做减法，对简单任务指定少agent，节省成本。</li><li>Tool design and selection are critical.<br />Tool Description要够好，否则Agent可能不会调用需要的MCP工具。</li><li>Let agents improve themselves.<br />使用tool-testing agent，让agent改进失败的prompt和流程、重写工具描述等。</li><li>Start wide, then narrow down.<br />这一条是因为Agent自己的搜索词写的比较AI，太长了，返回的结果很少。需要提示AI使用宽泛的提示词，然后再窄化范围精确搜索。</li><li>Guide the thinking process.<br />这一步是打印日志，让AI把思考过程打成标记、大纲、ToDoList，这样方便修改。</li><li>Parallel tool calling transforms speed and performance.<br />主Agent平行分派任务给子Agent；子Agent并行调用Tools。</li></ol><h2 id="eval-agents"><a class="markdownIt-Anchor" href="#eval-agents"></a> Eval Agents</h2><p>Multi-Agents的过程可能每一次都不同，因此不能使用传统的评估方法。</p><ol><li>小样本评估。不要等到测试用例足够多才开始测试，边测试边修改效果更好。</li><li>LLM评估。给出判断标准（事实/引用准确性、完整性、来源质量、多余/无效工具调用…），让LLM量化评估（0.0~1.0打分）</li><li>人工检查遗漏。如AI是不是只使用SEO靠前的，而不是权威的网站。</li></ol><p>需要注意，Multi-Agents会产生涌现(Emergent Behaviors)，对Leader Agent的改动会影响Sub Agent。</p><p>Multi-Agent框架最好考虑下面几个方面：</p><ol><li>工作分工（规划、解释、执行、自愈、总结）</li><li>问题如何分割成子问题（确定可执行标准）</li><li>效率（时间预期、工具调用次数限制、Scaling rules）</li></ol><h2 id="production-challenges"><a class="markdownIt-Anchor" href="#production-challenges"></a> Production challenges</h2><ol><li>Agent有状态，重构Agent影响很大，最好加上自愈Agent、错误处理系统。<br />此外还可以加上check point，一步一步来，失败了从这一步开始重新生成；而不是丢失上下文从头开始。</li><li>Agent的错误是“复利”的，前面错了会导致最后错得离谱。</li></ol><h3 id="debugging"><a class="markdownIt-Anchor" href="#debugging"></a> Debugging</h3><p>监控Agent的决策模式和交互结构，做到生产级追踪，更能系统性诊断和解决问题。</p><h3 id="deploy"><a class="markdownIt-Anchor" href="#deploy"></a> Deploy</h3><p>使用彩虹部署。旧会话分配到旧机器上，逐渐分配流量到新机器上，渐进替代，减少prompt改动的影响。</p><h3 id="sync-and-block"><a class="markdownIt-Anchor" href="#sync-and-block"></a> Sync and Block</h3><p>Leader Agent并行地分配任务给Sub tasks，但是实际上是以最后一个执行完的Sub Agent为准进行信息交互。这会造成等待与阻塞。但是如果Sub Agent分别处理每一个Sub Agent，又会出现上下文不共享的问题，局部扰乱整体。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;dont-build-multi-agents&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#dont-build-multi-agents&quot;&gt;&lt;/a&gt; Don’t Build Multi-Agents&lt;/h1&gt;
&lt;p&gt;&lt;a href=</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>慢sql优化</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/%E6%85%A2sql%E4%BC%98%E5%8C%96/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/%E6%85%A2sql%E4%BC%98%E5%8C%96/</id>
    <published>2025-06-12T16:00:00.000Z</published>
    <updated>2025-06-13T01:59:27.006Z</updated>
    
    <content type="html"><![CDATA[<h1 id="数据分页优化"><a class="markdownIt-Anchor" href="#数据分页优化"></a> 数据分页优化</h1><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> your_table <span class="keyword">where</span> type <span class="operator">=</span> ? limit <span class="keyword">start</span>, <span class="keyword">end</span>;</span><br></pre></td></tr></table></figure><p><code>limit</code>的分页方式是查出select的所有数据，然后舍弃<code>start</code>之前的数据。因此对于大数据量，性能很低。</p><h2 id="优化方案"><a class="markdownIt-Anchor" href="#优化方案"></a> 优化方案</h2><h3 id="偏移id"><a class="markdownIt-Anchor" href="#偏移id"></a> 偏移ID</h3><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="comment">-- 深分页慢sql，51sec</span></span><br><span class="line"><span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> emp <span class="keyword">where</span> ename<span class="operator">=</span><span class="string">&#x27;svZLER&#x27;</span> limit <span class="number">1000000</span>, <span class="number">10</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 使用id回表优化查询，44sec</span></span><br><span class="line"><span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> emp <span class="keyword">where</span> id <span class="keyword">in</span> (<span class="keyword">select</span> id <span class="keyword">from</span> emp <span class="keyword">where</span> ename<span class="operator">=</span><span class="string">&#x27;eMxdWz&#x27;</span>) limit <span class="number">1000000</span>, <span class="number">10</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">-- 子查询使用二级索引深分页，然后回表，37sec</span></span><br><span class="line"><span class="keyword">select</span> <span class="operator">*</span> <span class="keyword">from</span> emp <span class="keyword">inner</span> <span class="keyword">join</span> (<span class="keyword">select</span> id <span class="keyword">from</span> emp <span class="keyword">where</span> ename<span class="operator">=</span><span class="string">&#x27;eMxdWz&#x27;</span> limit <span class="number">1000000</span>, <span class="number">10</span>) b <span class="keyword">using</span>(id) ;</span><br><span class="line"><span class="comment">-- b using(id) 相当于 on b.id = emp.id</span></span><br></pre></td></tr></table></figure><h3 id="分段查询"><a class="markdownIt-Anchor" href="#分段查询"></a> 分段查询</h3>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;数据分页优化&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#数据分页优化&quot;&gt;&lt;/a&gt; 数据分页优化&lt;/h1&gt;
&lt;figure class=&quot;highlight sql&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>RPC</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/RPC/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/RPC/</id>
    <published>2025-06-09T16:00:00.000Z</published>
    <updated>2025-06-20T09:07:55.548Z</updated>
    
    <content type="html"><![CDATA[<h1 id="remote-procedure-call"><a class="markdownIt-Anchor" href="#remote-procedure-call"></a> Remote Procedure Call</h1><p>本地函数放到服务器运行，会出现若干问题：</p><ol><li>我怎么知道是哪个函数？Call Id<br />本地函数调用，可以直接用指针找到函数；但是远程过程调用不行。<br />因此我们需要分别在Client和Server维护一个“函数 &lt;-&gt; Call Id”的映射来确定所调用的函数。</li><li>Client如何将参数传送到Server？序列化与反序列化<br />本地函数调用，参数会压入栈；然而在远程过程调用中，Client与Server是不同的进程、处理器、操作系统、大小端，而且链表、对象这样的数据内存不分配在一处，加上网络传输必须要有容错机制，不能通过内存传递参数。<br />因此我们需要使用网络传输，Client要将参数转换为字节流，传输到Server后，再反序列化还原为参数。<br />这里还会涉及到数据格式的问题，JSON（性能不高）、XML、Protobuf、Thrift都是数据格式。</li><li>不使用内存，如何传输？网络传输<br />网络传输层需要将Call Id与字节流传输给Server，因此RPC基于传输层TCP协议，gRPC基于HTTP2协议（同样基于TCP）。</li></ol><blockquote><p>早期的RPC不使用HTTP，是因为当时HTTP不能建立长连接，并且HTTP头部过长且不能压缩。HTTP2解决了上述问题。</p></blockquote><h2 id="一个http请求"><a class="markdownIt-Anchor" href="#一个http请求"></a> 一个HTTP请求</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">http://localhost:8080/add?a=1&amp;b=2</span><br><span class="line">&quot;Content-Type&quot;: &quot;application/json&quot;</span><br></pre></td></tr></table></figure><p>这个请求指定了方法add、协议http、数据格式JSON</p><span id="more"></span><h1 id="grpc"><a class="markdownIt-Anchor" href="#grpc"></a> gRPC</h1><h2 id="protobuf"><a class="markdownIt-Anchor" href="#protobuf"></a> Protobuf</h2><p>Protocol Buffer，性能优于XML、JSON。</p><ul><li>压缩性能、序列化、传输速度快</li><li>向后兼容（不破坏旧接口）、加密性好（二进制）</li><li>Protobuf需要专门的解析器；只有通过proto文件才能了解数据结构</li></ul><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">message </span><span class="title class_">HelloReq</span> &#123;</span><br><span class="line">    <span class="keyword">message </span><span class="title class_">InnerReq</span> &#123;</span><br><span class="line">        <span class="type">string</span> name = <span class="number">1</span>;</span><br><span class="line">        <span class="type">string</span> url = <span class="number">2</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    Gender gender = <span class="number">1</span>;</span><br><span class="line">    map&lt;<span class="type">string</span>, <span class="type">string</span>&gt; map = <span class="number">2</span>;</span><br><span class="line">    google.protobuf.Timestamp createTime = <span class="number">3</span>;</span><br><span class="line">    <span class="keyword">repeated</span> InnerReq req = <span class="number">4</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">// -I 路径； --go_out 生成go代码； plugins=grpc:. 使用grpc拓展，使用grpc拓展生成接口代码，放在当前目录下</span><br><span class="line">protoc -I . &lt;filename&gt;.proto --go_out=plugins=grpc:.</span><br></pre></td></tr></table></figure><h2 id="proto2go"><a class="markdownIt-Anchor" href="#proto2go"></a> proto2go</h2><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">service </span><span class="title class_">Greeter</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> SayHello (HelloReq) <span class="keyword">returns</span> (HelloResp)</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>Server</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> GreeterServer <span class="keyword">interface</span> &#123;</span><br><span class="line">    SayHello(context.Context, *HelloReq) (*HelloReply, <span class="type">error</span>)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">RegisterGreeterServer</span><span class="params">(s *gprc.Server, srv GreeterServer)</span></span> &#123;</span><br><span class="line">    s.RegisterService(&amp;_Greeter_serviceDesc, srv)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>Client</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> greeterClient <span class="keyword">struct</span> &#123;</span><br><span class="line">    cc grpc.ClientConnInterface</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">NewGreeterClient</span><span class="params">(cc grpc.ClientConnInterface)</span></span> GreeterClient &#123;</span><br><span class="line">    <span class="comment">// 返回一个实现了Interface所有方法的结构体</span></span><br><span class="line">    <span class="keyword">return</span> &amp;greeterClient&#123;cc&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> GreeterClient <span class="keyword">interface</span> &#123;</span><br><span class="line">    SayHello(ctx content.Context, in *HelloReq, opts ...grpc.CallOption) (*HelloReply, <span class="type">error</span>)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(c *greeterClient)</span></span> SayHello(ctx context.Context, in *HelloReq, opts ...grpc.CallOption) (*HelloReply, <span class="type">error</span>) &#123;</span><br><span class="line">    out := <span class="built_in">new</span>(HelloReply)</span><br><span class="line">    err := c.cc.Invoke(ctx, <span class="string">&quot;/Greeter/SayHello&quot;</span>, in, out, opts...)</span><br><span class="line">    <span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">        <span class="keyword">return</span> out, <span class="literal">nil</span></span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="stream"><a class="markdownIt-Anchor" href="#stream"></a> Stream</h2><h3 id="simple-rpc"><a class="markdownIt-Anchor" href="#simple-rpc"></a> Simple RPC</h3><p>Client和Server都建立短连接。</p><h3 id="server-side-streaming-rpc"><a class="markdownIt-Anchor" href="#server-side-streaming-rpc"></a> Server-side streaming RPC</h3><p>Client发送1次请求，Server返回一段连续的Stream。<br />例如，Client发送一个股票代码，Server连续发送实时的K线数据。</p><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">service </span><span class="title class_">Greeter</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> GetStream(StreamReq) <span class="keyword">returns</span> (stream StreamResp)</span></span><br><span class="line"><span class="function">&#125;</span></span><br></pre></td></tr></table></figure><h3 id="client-side-streaming-rpc"><a class="markdownIt-Anchor" href="#client-side-streaming-rpc"></a> Client-side streaming RPC</h3><p>与Server-side相反。<br />例如，Server向Client请求当前室温，物联网终端Client不断向Server发送实时室温。</p><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">service </span><span class="title class_">Greeter</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> GetStream(stream StreamReq) <span class="keyword">returns</span> (StreamResp)</span></span><br><span class="line"><span class="function">&#125;</span></span><br></pre></td></tr></table></figure><h3 id="bidirectional-streaming-rpc"><a class="markdownIt-Anchor" href="#bidirectional-streaming-rpc"></a> Bidirectional streaming RPC</h3><p>Client与Server都可以向对方发送数据流，即实时交互。例如Chat Bot。</p><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">service </span><span class="title class_">Greeter</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> GetStream(stream StreamReq) <span class="keyword">returns</span> (stream StreamResp)</span></span><br><span class="line"><span class="function">&#125;</span></span><br></pre></td></tr></table></figure><h2 id="metadata"><a class="markdownIt-Anchor" href="#metadata"></a> MetaData</h2><p>gRPC和HTTP一样，可以携带一些MetaData</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">:authority [localhost:port]</span><br><span class="line">content-type [application/grpc]</span><br><span class="line">user-agent [grpc-gp/version]</span><br><span class="line"></span><br><span class="line">data [your_data]</span><br></pre></td></tr></table></figure><h1 id="interceptor"><a class="markdownIt-Anchor" href="#interceptor"></a> Interceptor</h1><figure class="highlight go"><table><tr><td class="code"><pre><span class="line">interceptorCust := <span class="function"><span class="keyword">func</span><span class="params">(ctx context.Context, req Interface&#123;&#125;, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler)</span></span> (resp Interface&#123;&#125;, err <span class="type">error</span>) &#123;</span><br><span class="line">    fmt.Println(<span class="string">&quot;接收到新请求: &quot;</span>, req)</span><br><span class="line">    start := time.Now()</span><br><span class="line"></span><br><span class="line">    res, err := handler(ctx, req)</span><br><span class="line"></span><br><span class="line">    fmt.Println(<span class="string">&quot;请求完成，耗时: &quot;</span>, time.Since(start))</span><br><span class="line">    <span class="keyword">return</span> res, err</span><br><span class="line">&#125;</span><br><span class="line">opt := grpc.UnaryInterceptor(interceptorCust)</span><br><span class="line">g := grpc.NewServer(opt)</span><br></pre></td></tr></table></figure><h1 id="validation"><a class="markdownIt-Anchor" href="#validation"></a> Validation</h1><p>plugin: protoc-gen-validate</p><figure class="highlight protobuf"><table><tr><td class="code"><pre><span class="line"><span class="keyword">message </span><span class="title class_">Person</span> &#123;</span><br><span class="line">    <span class="comment">// id &gt; 999</span></span><br><span class="line">    <span class="type">uint64</span> id = <span class="number">1</span> [(validate.rules).<span class="type">uint64</span>.gt = <span class="number">999</span>];</span><br><span class="line">    <span class="comment">// email validation</span></span><br><span class="line">    <span class="type">string</span> email = <span class="number">2</span> [(validate.rules).<span class="type">string</span>.email = <span class="literal">true</span>];</span><br><span class="line">    <span class="comment">// custom validation</span></span><br><span class="line">    <span class="type">string</span> name = <span class="number">3</span> [(validate.rules).<span class="type">string</span> = &#123;</span><br><span class="line">                        pattern: <span class="string">&quot;^[0-9]&amp;&quot;</span>,</span><br><span class="line">                        max_bytes: <span class="number">256</span>,</span><br><span class="line">                    &#125;];</span><br><span class="line">    <span class="comment">// not null</span></span><br><span class="line">    Location home = <span class="number">4</span> [(validate.rules).message.<span class="keyword">required</span> = ture];</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">message </span><span class="title class_">Location</span> &#123;</span><br><span class="line">    <span class="comment">// multi-args validation</span></span><br><span class="line">    <span class="type">double</span> lat = <span class="number">1</span> [(validate.rules).<span class="type">double</span> = &#123; gte: -<span class="number">90</span>, lte: <span class="number">90</span> &#125;];</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">protoc --validate_out=&quot;lang=go:.&quot;</span><br></pre></td></tr></table></figure><figure class="highlight go"><table><tr><td class="code"><pre><span class="line">p := <span class="built_in">new</span>(Person)</span><br><span class="line"><span class="comment">// throw error automatically</span></span><br><span class="line">err := p.Validate()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">    <span class="built_in">panic</span>(err)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>搭配拦截器</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">interceptor</span><span class="params">(ctx, req, info, handler)</span></span> (resp, err) &#123;</span><br><span class="line">    <span class="keyword">if</span> r, ok := req.(Validator); ok &#123;</span><br><span class="line">        <span class="keyword">if</span> err := r.Validate(); err != <span class="literal">nil</span> &#123;</span><br><span class="line">            <span class="keyword">return</span> <span class="literal">nil</span>, status.Error(codes.InvalidArgument, err.Error())</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> handler(ctx, req)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;remote-procedure-call&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#remote-procedure-call&quot;&gt;&lt;/a&gt; Remote Procedure Call&lt;/h1&gt;
&lt;p&gt;本地函数放到服务器运行，会出现若干问题：&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;我怎么知道是哪个函数？Call Id&lt;br /&gt;
本地函数调用，可以直接用指针找到函数；但是远程过程调用不行。&lt;br /&gt;
因此我们需要分别在Client和Server维护一个“函数 &amp;lt;-&amp;gt; Call Id”的映射来确定所调用的函数。&lt;/li&gt;
&lt;li&gt;Client如何将参数传送到Server？序列化与反序列化&lt;br /&gt;
本地函数调用，参数会压入栈；然而在远程过程调用中，Client与Server是不同的进程、处理器、操作系统、大小端，而且链表、对象这样的数据内存不分配在一处，加上网络传输必须要有容错机制，不能通过内存传递参数。&lt;br /&gt;
因此我们需要使用网络传输，Client要将参数转换为字节流，传输到Server后，再反序列化还原为参数。&lt;br /&gt;
这里还会涉及到数据格式的问题，JSON（性能不高）、XML、Protobuf、Thrift都是数据格式。&lt;/li&gt;
&lt;li&gt;不使用内存，如何传输？网络传输&lt;br /&gt;
网络传输层需要将Call Id与字节流传输给Server，因此RPC基于传输层TCP协议，gRPC基于HTTP2协议（同样基于TCP）。&lt;/li&gt;
&lt;/ol&gt;
&lt;blockquote&gt;
&lt;p&gt;早期的RPC不使用HTTP，是因为当时HTTP不能建立长连接，并且HTTP头部过长且不能压缩。HTTP2解决了上述问题。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;一个http请求&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#一个http请求&quot;&gt;&lt;/a&gt; 一个HTTP请求&lt;/h2&gt;
&lt;figure class=&quot;highlight plaintext&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;http://localhost:8080/add?a=1&amp;amp;b=2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;quot;Content-Type&amp;quot;: &amp;quot;application/json&amp;quot;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;这个请求指定了方法add、协议http、数据格式JSON&lt;/p&gt;</summary>
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>Advanced Go</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Advanced%20Go/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Advanced%20Go/</id>
    <published>2025-05-25T16:00:00.000Z</published>
    <updated>2025-06-02T06:49:22.251Z</updated>
    
    <content type="html"><![CDATA[<h1 id="gmp"><a class="markdownIt-Anchor" href="#gmp"></a> GMP</h1><h2 id="协程"><a class="markdownIt-Anchor" href="#协程"></a> 协程</h2><blockquote><p>协程是用户态的概念。多个协程实际上映射为1个线程。</p></blockquote><p>协程是用户态概念，因此创建、销毁、调度都在用户态完成，不需要切换内核态。<br />由于协程从属于同一个内核级线程，因此实际上无法并行；而一个协程的阻塞最终也会导致整个线程下的所有协程阻塞。</p><h2 id="goroutine"><a class="markdownIt-Anchor" href="#goroutine"></a> Goroutine</h2><blockquote><p>Go解耦了协程和线程的绑定关系，从而使线程变为一个中间层，协程可以灵活地映射到不同的线程上，相当于“虚拟线程”。</p></blockquote><p>好处如下：</p><ul><li>可以利用多个线程，实现并行</li><li>通过调度器，实现灵活的映射</li><li>栈空间动态扩展（线程大小固定，会产生内存浪费）</li></ul><h2 id="gmp-2"><a class="markdownIt-Anchor" href="#gmp-2"></a> GMP</h2><p>Goroutine Machine Processor<br />GMP就是协程调度器。<br />GMP有一个全局队列存储Goroutine；不过实际上Processor都会优先在自己的本地队列调度Goroutine（没有则向全局队列获取），并映射Goroutine到Machine上执行。<br />如果全局队列没有Goroutine，那么会尝试获取就绪态（正在IO）的协程。<br />如果仍然失败，那么会从其他Processor中窃取一半的Goroutine，实现负载均衡。</p><p>全局队列是互斥的，获取Goroutine要防止获取多次。</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> schedt <span class="keyword">struct</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    lock     mutex</span><br><span class="line">    runq     gQueue</span><br><span class="line">    runqsize <span class="type">int32</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><span id="more"></span><h3 id="g"><a class="markdownIt-Anchor" href="#g"></a> G</h3><p>Goroutine需要绑定到Processor才能运行，Processor就是对CPU资源的抽象。</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> g <span class="keyword">struct</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    m *m <span class="comment">// g与m映射</span></span><br><span class="line">    sched gobuf</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> gobuf <span class="keyword">struct</span> &#123;</span><br><span class="line">    sp  <span class="type">uintptr</span></span><br><span class="line">    pc  <span class="type">uintptr</span></span><br><span class="line">    ret <span class="type">uintptr</span></span><br><span class="line">    bp  <span class="type">uintptr</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="m"><a class="markdownIt-Anchor" href="#m"></a> M</h3><p>Machine是对线程的抽象。<br />Machine不能直接执行Goroutine，而需要首先与Processor绑定，由Processor实现代理。<br />同时，由于Processor中间层的存在，Goroutine与Machine不是紧耦合的，Goroutine完全可以跨Machine运行。</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> m <span class="keyword">struct</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    g0 *g <span class="comment">// Goroutine，特殊的协程调度，与m一对一绑定，负责执行g之间的切换调度</span></span><br><span class="line">    tls   <span class="comment">// Thread Local Storage，m.tls[0]存储当前运行的g</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="p"><a class="markdownIt-Anchor" href="#p"></a> P</h3><p>Processor是Golang的调度器。Processor代理Machine执行，提供一个透明（不可见）的调度机制。Processor的数量决定了Goroutine的并行程度。（当然，最终由CPU核数决定）</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">type</span> p <span class="keyword">struct</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// Head of Queue</span></span><br><span class="line">    runqhead <span class="type">uint32</span></span><br><span class="line">    runqtail <span class="type">uint32</span></span><br><span class="line">    runq     [<span class="number">256</span>]guintptr <span class="comment">// Runnable Goroutine Queue</span></span><br><span class="line"></span><br><span class="line">    runnext guintptr <span class="comment">// 下一个Runnable状态的Goroutine</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="g0与g的转换"><a class="markdownIt-Anchor" href="#g0与g的转换"></a> g0与g的转换</h2><p>g0与m一对一绑定，负责执行g之间的切换调度</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="comment">// g0 -&gt; g，g0将执行权交给对应的g</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">gogo</span><span class="params">()</span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">// g -&gt; g0，g阻塞或协程切换使，交换控制权</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">m_call</span><span class="params">()</span></span></span><br></pre></td></tr></table></figure><h2 id="goroutine调度"><a class="markdownIt-Anchor" href="#goroutine调度"></a> Goroutine调度</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">g0 -&gt; schedule() -&gt; execute() -&gt; gogo() -&gt; g</span><br><span class="line"></span><br><span class="line">g -&gt; m_call()</span><br><span class="line">m_call() -&gt; gosched_m() -&gt; schedule()</span><br><span class="line"></span><br><span class="line">m_call() -&gt; park_m(): 暂停goroutine</span><br><span class="line">park_m() -&gt; schedule()</span><br><span class="line"></span><br><span class="line">m_call() -&gt; goexit0(): Monitor g</span><br><span class="line">goexit0() -&gt; schedule()</span><br></pre></td></tr></table></figure><h3 id="主动调度"><a class="markdownIt-Anchor" href="#主动调度"></a> 主动调度</h3><p>用户发起调度，主动执行让渡</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Gosched</span><span class="params">()</span></span> &#123;</span><br><span class="line">    checkTimeouts()</span><br><span class="line">    mcall(gosched_m)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="被动调度"><a class="markdownIt-Anchor" href="#被动调度"></a> 被动调度</h3><p>互斥锁、等待等状态</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 暂停goroutine，与processor解绑</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">gopark</span><span class="params">()</span></span></span><br><span class="line"><span class="comment">// 唤醒，processor优先运行唤醒goroutine</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">goready</span><span class="params">()</span></span></span><br></pre></td></tr></table></figure><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">gopark</span><span class="params">(unlockf <span class="keyword">func</span>(*g, unsafe.Pointer)</span></span> <span class="type">bool</span>, lock unsafe.Pointer, reason waitReason, traceEv <span class="type">byte</span>, traceskip <span class="type">int</span>) &#123;</span><br><span class="line">    mcall(park_m)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">park_m</span><span class="params">(gp *g)</span></span> &#123;</span><br><span class="line">    _g_ := getg()</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 改变Goroutine状态</span></span><br><span class="line">    casgstatus(gp, _Grunning, _Gwaiting)</span><br><span class="line">    <span class="comment">// 出队</span></span><br><span class="line">    dropg()</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 新一轮调度</span></span><br><span class="line">    schedule()</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">goready</span><span class="params">(gp *g, traceskip <span class="type">int</span>)</span></span> &#123;</span><br><span class="line">    systemstack(<span class="function"><span class="keyword">func</span><span class="params">()</span></span>) &#123;</span><br><span class="line">        ready(gp, traceskip, <span class="literal">true</span>)</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">ready</span><span class="params">(gp *g, traceskip <span class="type">int</span>, next <span class="type">bool</span>)</span></span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    _g_ := getg()</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// CAS状态切换</span></span><br><span class="line">    casgstatus(gp, _Gwaiting, _Grunnable)</span><br><span class="line">    <span class="comment">// 入队</span></span><br><span class="line">    runqput(_g_.m.p.ptr(), gp, next)</span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="正常调度"><a class="markdownIt-Anchor" href="#正常调度"></a> 正常调度</h3><p>Goroutine正常执行结束，通过<code>m_call()</code>返回控制权给g0</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Gosched</span><span class="params">()</span></span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    mcall(gosched_m)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">gosched_m</span><span class="params">(gp *g)</span></span> &#123;</span><br><span class="line">    goshedImpl(gp)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">goschedImpl</span><span class="params">(gp *g)</span></span> &#123;</span><br><span class="line">    status := readgstatus(gp)</span><br><span class="line">    <span class="keyword">if</span> (status&amp;^_Gscan != _Grunning) &#123;</span><br><span class="line">        dumpgstatus(gp)</span><br><span class="line">        throw(<span class="string">&quot;bad g status&quot;</span>)</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// CAS切换Goroutine状态</span></span><br><span class="line">    casgstatus(gp, _Grunning, _Grunnable)</span><br><span class="line">    <span class="comment">// 解绑当前Goroutine和Processor</span></span><br><span class="line">    dropg()</span><br><span class="line">    <span class="comment">// 加锁入队全局队列</span></span><br><span class="line">    lock(&amp;sched.lock)</span><br><span class="line">    globrunqput(gp)</span><br><span class="line">    unlock(&amp;sched.lock)</span><br><span class="line"></span><br><span class="line">    schedule()</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="抢占调度"><a class="markdownIt-Anchor" href="#抢占调度"></a> 抢占调度</h3><p>Monitor g全局监控完成<br />如果某个Goroutine发起系统调用，并过长时间占据Processor（如恶意抢占系统资源），Monitor g将会转移这个Goroutine所在的Processor与Machine的绑定，从而避免该Processor的阻塞。<br />注意，Monitor g并没有办法中断系统调用中的Goroutine（此时已经在内核态）。</p><h2 id="schedule"><a class="markdownIt-Anchor" href="#schedule"></a> schedule()</h2><p>调度流程主干方法：</p><ol><li>寻找下一个可执行的Goroutine</li><li>执行Goroutine</li></ol><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">schedule</span><span class="params">()</span></span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    gp, inheritTime, tryWakeP := findRunnable()</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">    execute(gp, inheritTime)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="findrunnable"><a class="markdownIt-Anchor" href="#findrunnable"></a> findRunnable()</h3><p>为了防止Processor过于繁忙，全局队列的Goroutine饿死，每61次调度后Processor就会优先从全局队列取Goroutine。<br />此时，如果本地Processor队列满了，会将本地Goroutine踢出，以换取全局Goroutine入队，负载均衡。<br />如果全局队列是空的，会获取因为IO操作而处于就绪态的Goroutine。<br />如果没有获取到IO中的Goroutine，当前Processor将会为其他Processor负载均衡，获取其他Processor队列中一半的Goroutine到本地队列。</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">findRunnable</span><span class="params">()</span></span> (gp *g, inheritTime, tryWakeP <span class="type">bool</span>) &#123;</span><br><span class="line">    _g_ := getg()</span><br><span class="line"></span><br><span class="line">top:</span><br><span class="line">    _p_ := _g_.m.p.ptr()</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 每61次调度，优先从全局队列获取</span></span><br><span class="line">    <span class="keyword">if</span> _p_.schedtick % <span class="number">61</span> == <span class="number">0</span> &amp;&amp; sched.runqsize &gt; <span class="number">0</span> &#123;</span><br><span class="line">        lock(&amp;sched.lock)</span><br><span class="line">        gp = globrunqget(_p_, <span class="number">1</span>)</span><br><span class="line">        unlock(&amp;sched.lock)</span><br><span class="line">        <span class="keyword">if</span> gp != <span class="literal">nil</span> &#123;</span><br><span class="line">            <span class="keyword">return</span> gp, <span class="literal">false</span>, <span class="literal">false</span></span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 正常情况下，从当前Processor本地队列获取Goroutine</span></span><br><span class="line">    <span class="comment">// go特殊语法，初始化后判断</span></span><br><span class="line">    <span class="keyword">if</span> gp, inheritTime := runqget(_p_); gp != <span class="literal">nil</span> &#123;</span><br><span class="line">        <span class="keyword">return</span> gp, inheritTime, <span class="literal">false</span></span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 本地Processor队列没有获取成功，未返回（如队列为空）</span></span><br><span class="line">    <span class="comment">// 此时尝试从全局队列获取Goroutine</span></span><br><span class="line">    <span class="keyword">if</span> sched.runqsize != <span class="number">0</span> &#123;</span><br><span class="line">        lock(&amp;sched.lock)</span><br><span class="line">        gp = globrunqget(_p_, <span class="number">0</span>)</span><br><span class="line">        unlock(&amp;sched.lock)</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 全局队列获取Goroutine失败</span></span><br><span class="line">    <span class="comment">// 尝试从IO流获取</span></span><br><span class="line">    <span class="keyword">if</span> netpollinited() &amp;&amp; atomic.Load(&amp;netpollWaiters) &gt; <span class="number">0</span> &amp;&amp; atomic.Load64(&amp;sched.lastpoll != <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> list := netpoll(<span class="number">0</span>); !list.empty() &#123;</span><br><span class="line">            gp := list.pop()</span><br><span class="line">            injectglist(&amp;list)</span><br><span class="line">            casgstatus(gp, _Gwaiting, +Grunnable)</span><br><span class="line">            <span class="keyword">return</span> gp, <span class="literal">false</span>, <span class="literal">false</span></span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 从IO流获取失败，未返回</span></span><br><span class="line">    procs := <span class="type">uint32</span>(gomaxprocs)</span><br><span class="line">    <span class="keyword">if</span> _g_.m.spinning || <span class="number">2</span>*atomic.Load(&amp;sched.nmspinning) &lt; procs-atomic.Load(&amp;sched.npidle) &#123;</span><br><span class="line">        <span class="keyword">if</span> !_g_.m.spinning &#123;</span><br><span class="line">            _g_.m.spinning = <span class="literal">true</span></span><br><span class="line">            atomic.Xadd(&amp;sched.nmspinning, <span class="number">1</span>)</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="comment">// 负载均衡，从其他Processor中获取一半Goroutine</span></span><br><span class="line">        gp, inheritTime, tnow, w, newWork := stealWork(now)</span><br><span class="line">        now = tnow</span><br><span class="line">        <span class="keyword">if</span> gp != <span class="literal">nil</span> &#123;</span><br><span class="line">            <span class="keyword">return</span> gp, inheritTime, <span class="literal">false</span></span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">if</span> newWork &#123;</span><br><span class="line">            <span class="keyword">goto</span> top</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">if</span> w != <span class="number">0</span> &amp;&amp; (pollUntil == <span class="number">0</span> || w &lt; pollUntil) &#123;</span><br><span class="line">            pollUntil = w</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h4 id="stealwork"><a class="markdownIt-Anchor" href="#stealwork"></a> stealWork()</h4><p>负载均衡，从其他Processor中获取Goroutine时，stealWork</p><ul><li>最多遍历4次队列。其中一次成功就会return</li><li>每一次尝试获取Processor之前，都会对队列局部加锁（锁住队列头和队尾即可）</li></ul><h3 id="execute"><a class="markdownIt-Anchor" href="#execute"></a> execute()</h3><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">execute</span><span class="params">(gp *g, inheritTime <span class="type">bool</span>)</span></span> &#123;</span><br><span class="line">    _g_ := getg()</span><br><span class="line"></span><br><span class="line">    _g_.m.curg = gp</span><br><span class="line">    <span class="comment">// 映射Processor与Machine</span></span><br><span class="line">    gp.m = _g_.m</span><br><span class="line">    <span class="comment">// CAS切换状态</span></span><br><span class="line">    casgstatus(gp, _Grunnable, _Grunning)</span><br><span class="line">    gp.waitsince = <span class="number">0</span></span><br><span class="line">    gp.preempt = <span class="literal">false</span></span><br><span class="line">    gp.stackguard0 = gp.stack.lo + _StackGuard</span><br><span class="line">    <span class="keyword">if</span> !inheritTime &#123;</span><br><span class="line">        <span class="comment">// 更新调度次数</span></span><br><span class="line">        _g_.m.p.ptr().schedtick++</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 执行Goroutine任务</span></span><br><span class="line">    gogo(&amp;gp.sched)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h1 id="geohash"><a class="markdownIt-Anchor" href="#geohash"></a> GeoHash</h1>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;gmp&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#gmp&quot;&gt;&lt;/a&gt; GMP&lt;/h1&gt;
&lt;h2 id=&quot;协程&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#协程&quot;&gt;&lt;/a&gt; 协程&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;协程是用户态的概念。多个协程实际上映射为1个线程。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;协程是用户态概念，因此创建、销毁、调度都在用户态完成，不需要切换内核态。&lt;br /&gt;
由于协程从属于同一个内核级线程，因此实际上无法并行；而一个协程的阻塞最终也会导致整个线程下的所有协程阻塞。&lt;/p&gt;
&lt;h2 id=&quot;goroutine&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#goroutine&quot;&gt;&lt;/a&gt; Goroutine&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;Go解耦了协程和线程的绑定关系，从而使线程变为一个中间层，协程可以灵活地映射到不同的线程上，相当于“虚拟线程”。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;好处如下：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;可以利用多个线程，实现并行&lt;/li&gt;
&lt;li&gt;通过调度器，实现灵活的映射&lt;/li&gt;
&lt;li&gt;栈空间动态扩展（线程大小固定，会产生内存浪费）&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;gmp-2&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#gmp-2&quot;&gt;&lt;/a&gt; GMP&lt;/h2&gt;
&lt;p&gt;Goroutine Machine Processor&lt;br /&gt;
GMP就是协程调度器。&lt;br /&gt;
GMP有一个全局队列存储Goroutine；不过实际上Processor都会优先在自己的本地队列调度Goroutine（没有则向全局队列获取），并映射Goroutine到Machine上执行。&lt;br /&gt;
如果全局队列没有Goroutine，那么会尝试获取就绪态（正在IO）的协程。&lt;br /&gt;
如果仍然失败，那么会从其他Processor中窃取一半的Goroutine，实现负载均衡。&lt;/p&gt;
&lt;p&gt;全局队列是互斥的，获取Goroutine要防止获取多次。&lt;/p&gt;
&lt;figure class=&quot;highlight go&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;type&lt;/span&gt; schedt &lt;span class=&quot;keyword&quot;&gt;struct&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    ...&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    lock     mutex&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    runq     gQueue&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    runqsize &lt;span class=&quot;type&quot;&gt;int32&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;</summary>
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>Redis原理</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Redis%E5%8E%9F%E7%90%86/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Redis%E5%8E%9F%E7%90%86/</id>
    <published>2025-05-19T16:00:00.000Z</published>
    <updated>2025-07-10T00:52:55.587Z</updated>
    
    <content type="html"><![CDATA[<h1 id="跳表"><a class="markdownIt-Anchor" href="#跳表"></a> 跳表</h1><p>ZSet的实现方式有跳表、压缩列表。</p><ul><li>压缩列表：比较方便地搜索头节点和尾节点。数量&lt;128，所有元素长度&lt;64B时使用。</li><li>跳表：就是链表二分搜索的数据结构。多级链表，最高级链接的节点最稀疏。可以从高到低寻找，加快效率。</li></ul><blockquote><p>同样，跳表对范围查询支持较好，二分找到开头，然后遍历即可。</p></blockquote><h2 id="redis为什么不用b树mysql为什么不用跳表"><a class="markdownIt-Anchor" href="#redis为什么不用b树mysql为什么不用跳表"></a> Redis为什么不用b+树？MySQL为什么不用跳表？</h2><p>这个问题在于 Redis是直接操作内存的并不需要磁盘io而MySQL需要去读取io，所以mysql要使用b+树的方式减少磁盘io，B+树的原理是 叶子节点存储数据，非叶子节点存储索引，每次读取磁盘页时就会读取一整个节点,每个叶子节点还有指向前后节点的指针，为的是最大限度的降低磁盘的IO;因为数据在内存中读取耗费的时间是从磁盘的IO读取的百万分之一 而Redis是 内存中读取数据，不涉及IO，因此使用了跳表，跳表明显是更快更简单的方式。</p><h1 id="单线程网络io-kv读写"><a class="markdownIt-Anchor" href="#单线程网络io-kv读写"></a> 单线程网络IO、KV读写</h1><p>Redis的网络IO和KeyValue读写是由一个线程来完成的。<br />而Redis的持久化、异步删除、集群数据同步是额外的线程执行。</p><p>也由于Redis是单线程的，所以要特别小心耗时的操作，这些操作会阻塞后续指令。</p><blockquote><p>简单来说就是处理事务一套、前台接待一套。不会因为前面办事导致人均等待时间太久。</p></blockquote><p>Redis使用IO多路复用（epoll），将连接信息、事件放到队列中，使其能够处理并发的客户端连接。</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">socket: &#123;</span><br><span class="line">    s0</span><br><span class="line">    s1</span><br><span class="line">    s2</span><br><span class="line">    s3</span><br><span class="line">    &quot;...&quot;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">IO多路复用: &#123;</span><br><span class="line">    s3 -&gt; s2 -&gt; s1 -&gt; s0</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">事件处理器: &#123;</span><br><span class="line">    连接处理器</span><br><span class="line">    命令请求处理器</span><br><span class="line">    命令回复处理器</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">socket -&gt; IO多路复用 -&gt; 文件事件分派器 -&gt; 事件处理器</span><br></pre></td></tr></table></figure><h1 id="详解get-key"><a class="markdownIt-Anchor" href="#详解get-key"></a> 详解GET key</h1><p>Redis相当于HashMap，也由于Hash是无序的，因此<code>scan</code>这样的流式查询，在查改场景中，可能会漏扫中途插入到前面下标的元素。</p><h1 id="redis持久化"><a class="markdownIt-Anchor" href="#redis持久化"></a> Redis持久化</h1><h2 id="rdb-snapshot"><a class="markdownIt-Anchor" href="#rdb-snapshot"></a> RDB Snapshot</h2><p>默认情况下，Redis将内存数据快照保存为<code>dump.rdb</code>，可以使用</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">save &lt;time_duration&gt; &lt;row_insertion&gt;</span><br></pre></td></tr></table></figure><p>指示Redis多少秒内插入多少条数据后持久化到数据库<br />也可以直接用<code>save</code>和<code>bgsave</code>命令写入数据库</p><h3 id="bgsave-异步持久化"><a class="markdownIt-Anchor" href="#bgsave-异步持久化"></a> bgsave 异步持久化</h3><p>bgsave使用写时复制COW。bgsave从主线程fork出来，当主线程修改数据时，bgsave线程会将写入数据拷贝一份，然后写入rdb</p><h2 id="append-only-file"><a class="markdownIt-Anchor" href="#append-only-file"></a> Append-Only File</h2><p>快照不能做到完全持久，假如服务宕机，可能会丢失几条写入。<br />这时候我们直接做个命令日志AOF，将执行的修改指令写入<code>appendonly.aof</code>中</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">appendonly yes</span><br><span class="line">appendfilename &quot;appendonly.aof&quot;</span><br></pre></td></tr></table></figure><p>aof有三种模式<code>appendfsync</code>：</p><ul><li>always：立刻写入磁盘</li><li>everysec：每秒写一次</li><li>no：交给OS调度<br />但是，由于aof是记录命令，需要执行时间，对于持久化大量数据比较耗时间。<br />对于连续操作（如自增）aof会优化为1条命令，可以用<code>bgrewriteaof</code>命令手动重写</li></ul><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"># 最小重构大小</span><br><span class="line">auto-aof-rewrite-min-size 64mb</span><br><span class="line"># 增长了100%，即128mb就重构</span><br><span class="line">auto-aof-rewrite-percentage 100</span><br></pre></td></tr></table></figure><h2 id="redis4-混合持久化"><a class="markdownIt-Anchor" href="#redis4-混合持久化"></a> Redis4 混合持久化</h2><p>由于Redis重启时优先使用aof恢复数据，rdb利用率不高。因此出现了混合持久化</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"># 必须同时开启aof</span><br><span class="line">aof-use-rdb-preamle yes</span><br><span class="line"># 可以直接把快照关掉，因为混合持久化都写在aof里面</span><br></pre></td></tr></table></figure><p>开启后，当aof重写时，会直接写入rdb，将rdb快照和aof增量存储在一起。<br />于是Redis重启可以先读rdb，再执行增量aof恢复数据，提高效率。</p><h1 id="redis主从"><a class="markdownIt-Anchor" href="#redis主从"></a> Redis主从</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"># redis-&lt;your_port&gt;.conf</span><br><span class="line">pidfile /var/run/redis_&lt;your_port&gt;.pid</span><br><span class="line">logfile &quot;&lt;your_port&gt;.log&quot;</span><br><span class="line"># 数据存放目录</span><br><span class="line">dir /usr/local/redis/data/&lt;your_port&gt;</span><br><span class="line"></span><br><span class="line">### 主从复制</span><br><span class="line">replicaof &lt;main_redis_ip&gt; &lt;port&gt;</span><br><span class="line"># 从节点，只读</span><br><span class="line">replica-read-only yes</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">### 启动</span><br><span class="line"># 启动从节点</span><br><span class="line">redis-server redis-&lt;your_port&gt;.conf</span><br><span class="line"># 连接到从节点</span><br><span class="line">redis-cli -p &lt;minor_redis_port&gt;</span><br></pre></td></tr></table></figure><h2 id="主从原理"><a class="markdownIt-Anchor" href="#主从原理"></a> 主从原理</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">master: &#123;</span><br><span class="line">    rdb data</span><br><span class="line">    repl buffer</span><br><span class="line">&#125;</span><br><span class="line">slave</span><br><span class="line"></span><br><span class="line">slave -&gt; master: 1. psync全量复制同步数据（通过socket长连接）</span><br><span class="line">master.rdb data -&gt; master.rdb data: 2.1 收到psync命令，执行bgsave生成最新rdb快照</span><br><span class="line">master.repl buffer -&gt; master.repl buffer: 2.2 主节点将增量写语句更新到buffer</span><br><span class="line">master.rdb data -&gt; slave: 3. 发送rdb数据</span><br><span class="line">slave -&gt; slave: 4. 清空旧数据，加载主节点rdb</span><br><span class="line">master.repl buffer -&gt; slave: 5. 发送缓冲区写命令</span><br><span class="line">slave -&gt; slave: 6. 执行主节点buffer写命令</span><br><span class="line">master -&gt; slave: 7. 主节点通过socket长连接，持续发送写命令给从节点，保持数据一致</span><br></pre></td></tr></table></figure><h2 id="断点续传"><a class="markdownIt-Anchor" href="#断点续传"></a> 断点续传</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">master: &#123;</span><br><span class="line">    repl backlog buffer</span><br><span class="line">&#125;</span><br><span class="line">slave</span><br><span class="line"></span><br><span class="line">slave -&gt; master: 1. 连接断开</span><br><span class="line">master.repl backlog buffer -&gt; master.repl backlog buffer: 2. 主节点增量写命令写入buffer</span><br><span class="line">slave -&gt; master: 3. 恢复socket长连接</span><br><span class="line">slave -&gt; master: 4. psync(offset)带偏移量</span><br><span class="line">master -&gt; slave: 5. 若offset在buffer中，断点以后的数据发送给从节点；否则，全量发送</span><br><span class="line">master -&gt; slave: 6. 持续发送buffer写命令，保持数据一致</span><br></pre></td></tr></table></figure><p>如果存在很多从节点，那么主节点传输压力会比较大。可以采用树型架构，让从节点再给它的子节点传输数据。</p><h1 id="哨兵高可用"><a class="markdownIt-Anchor" href="#哨兵高可用"></a> 哨兵高可用</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">sentinel_cluster: &#123;</span><br><span class="line">    sentinel1 &lt;-&gt; sentinel2 &lt;-&gt; sentinel3 &lt;-&gt; sentinel1</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">client -&gt; master &lt;-&gt; sentinel_cluster</span><br><span class="line">master -&gt; slave1</span><br><span class="line">master -&gt; slave2</span><br><span class="line">client -&gt; sentinel_cluster</span><br><span class="line">sentinel_cluster &lt;-&gt; slave1</span><br><span class="line">sentinel_cluster &lt;-&gt; slave2</span><br></pre></td></tr></table></figure><p>哨兵会动态监听redis主节点，如果主节点挂了，哨兵会选择一个新redis示例作为主节点（通知给client端）</p><h2 id="开启哨兵"><a class="markdownIt-Anchor" href="#开启哨兵"></a> 开启哨兵</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"># sentinel.conf</span><br><span class="line"></span><br><span class="line">port 26379</span><br><span class="line">pidfile &lt;your_file&gt;</span><br><span class="line">logfile &lt;your_file&gt;</span><br><span class="line">dir &quot;&lt;your_dir&gt;&quot;</span><br><span class="line"></span><br><span class="line"># quorm是指多少个sentinel同时认为主节点挂了，才让master失效，一般设置为一半以上</span><br><span class="line">sentinel monitor mymaster &lt;redis_ip&gt; &lt;redis_port&gt; &lt;quorm&gt;</span><br></pre></td></tr></table></figure><p>启动哨兵<code>./redis-sentinel sentinel.conf</code></p><h1 id="redis-cluster"><a class="markdownIt-Anchor" href="#redis-cluster"></a> Redis Cluster</h1><p>当哨兵集群选举新节点的时候，服务会宕机几秒钟。因此我们需要Cluster</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">client1 -&gt; RedisCluster</span><br><span class="line">client2 -&gt; RedisCluster</span><br><span class="line">RedisCluster: Hash slot: CRC16(key) % 16384</span><br><span class="line">RedisCluster -&gt; Redis集群</span><br><span class="line">Redis集群: &#123;</span><br><span class="line">    master1 -&gt; slave1-1</span><br><span class="line">    master1 -&gt; slave1-2</span><br><span class="line">    </span><br><span class="line">    master2 -&gt; slave2-1</span><br><span class="line">    master2 -&gt; slave2-2</span><br><span class="line">    </span><br><span class="line">    master3 -&gt; slave3-1</span><br><span class="line">    master3 -&gt; slave3-2</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>在Cluster中，每个master数据是不重叠的，数据会被分片储存。通过Hash算法来决定存储数据到哪一个master节点。<br />使用Cluster，可以避免Redis服务完全宕机。<br />2的幂次取模小技巧：</p><p><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>X</mi><mspace></mspace><mspace width="0.6666666666666666em"/><mrow><mi mathvariant="normal">m</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">d</mi></mrow><mtext> </mtext><mtext> </mtext><msup><mn>2</mn><mi>n</mi></msup><mo>=</mo><mi>X</mi><mtext> &amp; </mtext><mo stretchy="false">(</mo><msup><mn>2</mn><mi>n</mi></msup><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">X \mod 2^n = X \text{ \&amp; } (2^n - 1)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.07847em;">X</span><span class="mspace allowbreak"></span><span class="mspace" style="margin-right:0.6666666666666666em;"></span></span><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord"><span class="mord"><span class="mord mathrm">m</span><span class="mord mathrm">o</span><span class="mord mathrm">d</span></span></span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord">2</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.664392em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.07847em;">X</span><span class="mord text"><span class="mord"> &amp; </span></span><span class="mopen">(</span><span class="mord"><span class="mord">2</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.664392em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span></span></span></span></p><h2 id="redis集群搭建"><a class="markdownIt-Anchor" href="#redis集群搭建"></a> Redis集群搭建</h2><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">redis-cluster/</span><br><span class="line">|-- 8000</span><br><span class="line">|   `-- redis.conf</span><br><span class="line">|-- 8010</span><br><span class="line">`-- 8020</span><br></pre></td></tr></table></figure><ol><li>Redis配置</li></ol><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line"># ...其他配置</span><br><span class="line"></span><br><span class="line">daemonize yes</span><br><span class="line">port 8000</span><br><span class="line">dir /path/to/redis-cluster/8000/</span><br><span class="line"># 启用集群</span><br><span class="line">cluster-enabled yes</span><br><span class="line">cluster-config-file nodes-8000.conf</span><br><span class="line">cluster-node-timeout 5000</span><br><span class="line"># 密码</span><br><span class="line">requirepass &lt;your_password&gt;</span><br><span class="line">masterauth &lt;your_auth_password&gt;</span><br></pre></td></tr></table></figure><ol start="2"><li>启动所有master和slave节点</li></ol><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">redis-server /path/to/redis-cluster/80*/redis.conf</span><br><span class="line">ps aux | grep redis</span><br></pre></td></tr></table></figure><ol start="3"><li>开启集群</li></ol><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">replicas表示节点的副本，配置为1，则1主1从</span></span><br><span class="line">redis-cli -a &lt;your_auth_password&gt; --cluster create --cluster-replicas 1 \</span><br><span class="line">localhost:8000 localhost:8001 localhost:8002 ...</span><br></pre></td></tr></table></figure><blockquote><p>注意，第二次启动集群后，就不需要这一步了。节点会自动读取<code>nodes-8000.conf</code>文件，恢复上次集群状态。</p></blockquote><ol start="4"><li>进入redis节点验证配置</li></ol><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">cluster info</span><br><span class="line">cluster nodes</span><br></pre></td></tr></table></figure><h1 id="redission原理"><a class="markdownIt-Anchor" href="#redission原理"></a> Redission原理</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Thread1: &#123;</span><br><span class="line">    Redission</span><br><span class="line">&#125;</span><br><span class="line">Thread2: &#123;</span><br><span class="line">    Redission</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">Thread1.Redission -&gt; Try Lock</span><br><span class="line">Try Lock -&gt; 守护线程: 加锁成功</span><br><span class="line">守护线程 -&gt; Redis(Master): lock，每隔10s检查线程是否仍持有锁。如果持有，则延长锁失效时间</span><br><span class="line"></span><br><span class="line">Thread2.Redission -&gt; Try Lock</span><br><span class="line">Try Lock -&gt; Thread2.Redission: 加锁失败，使用while自旋尝试加锁</span><br></pre></td></tr></table></figure><p>Redission利用了Redis Lua脚本保证原子操作。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;跳表&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#跳表&quot;&gt;&lt;/a&gt; 跳表&lt;/h1&gt;
&lt;p&gt;ZSet的实现方式有跳表、压缩列表。&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;压缩列表：比较方便地搜索头节点和尾节点。数量&amp;lt;128，所有元素长度&amp;lt;</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>Hertz使用</title>
    <link href="http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/Hertz%E4%BD%BF%E7%94%A8/"/>
    <id>http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/Hertz%E4%BD%BF%E7%94%A8/</id>
    <published>2025-05-19T16:00:00.000Z</published>
    <updated>2025-05-31T02:21:11.841Z</updated>
    
    <content type="html"><![CDATA[<h1 id="hello-world"><a class="markdownIt-Anchor" href="#hello-world"></a> Hello World</h1><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">    h := server.Default()</span><br><span class="line"></span><br><span class="line">    h.GET(<span class="string">&quot;/hello&quot;</span>, <span class="function"><span class="keyword">func</span><span class="params">(c context.Context, ctx *app.RequestContext)</span></span> &#123;</span><br><span class="line">        ctx.Data(consts.StatusOK, consts.MIMETextPlain, []<span class="type">byte</span>(<span class="string">&quot;Hello World!&quot;</span>))</span><br><span class="line">    &#125;)</span><br><span class="line"></span><br><span class="line">    h.Spin()</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">go run main.go</span></span><br></pre></td></tr></table></figure><h1 id="idl"><a class="markdownIt-Anchor" href="#idl"></a> IDL</h1><p>Thrift</p><figure class="highlight thrift"><table><tr><td class="code"><pre><span class="line"># echo.thrift</span><br><span class="line"><span class="keyword">namespace</span> go api</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Request</span> </span>&#123;</span><br><span class="line">    <span class="number">1</span>: <span class="type">string</span> message</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Response</span> </span>&#123;</span><br><span class="line">    <span class="number">1</span>: <span class="type">string</span> message</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">service</span> <span class="title">Echo</span> </span>&#123;</span><br><span class="line">    Response echo(<span class="number">1</span>: Request req)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>CloudweGo代码生成</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">go install github.com/cloudwego/thriftgo@latest</span><br><span class="line"></span><br><span class="line">mkdir -p demo/demo_thrift</span><br><span class="line">cd demo/demo_thrift</span><br><span class="line">cwgo server --type RPC \</span><br><span class="line">--module demo/demo_thrift \</span><br><span class="line">--service demo_thrift \</span><br><span class="line">--idl ../../echo.thrift</span><br></pre></td></tr></table></figure><p>Protobuf</p><figure class="highlight proto"><table><tr><td class="code"><pre><span class="line">syntax = <span class="string">&quot;proto3&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">package</span> pbapi;</span><br><span class="line"></span><br><span class="line"><span class="keyword">option</span> go_package = <span class="string">&quot;/pbapi&quot;</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">message </span><span class="title class_">Request</span> &#123;</span><br><span class="line">    <span class="type">string</span> msg = <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">message </span><span class="title class_">Response</span> &#123;</span><br><span class="line">    <span class="type">string</span> msg = <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">service </span><span class="title class_">EchoService</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> Echo (Request) <span class="keyword">returns</span> (Response) </span>&#123;&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>CloudweGo代码生成</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">mkdir -p demo/demo_proto</span><br><span class="line">cd demo/demo_proto</span><br><span class="line"></span><br><span class="line">cwgo server -I ../../idl</span><br><span class="line">--type RPC \</span><br><span class="line">--module demo/demo_proto \</span><br><span class="line">--service demo_proto \</span><br><span class="line">--idl ../../echo.thrift</span><br></pre></td></tr></table></figure><h2 id="makefile自动cwgo代码生成"><a class="markdownIt-Anchor" href="#makefile自动cwgo代码生成"></a> MakeFile自动cwgo代码生成</h2><figure class="highlight make"><table><tr><td class="code"><pre><span class="line"><span class="meta"><span class="keyword">.PHONY</span>: gen-demo-proto</span></span><br><span class="line"><span class="section">gen-demo-proto:</span></span><br><span class="line">    @cd demo/demo_proto &amp;&amp; cwgo server -I ../../idl --type RPC --module demo/demo_proto --service demo_proto --idl ../../echo.thrift</span><br></pre></td></tr></table></figure><h1 id="consul服务注册-发现"><a class="markdownIt-Anchor" href="#consul服务注册-发现"></a> Consul服务注册、发现</h1><p>服务注册用于为服务集群提供统一接口，自动处理集群loadbalance和宕机</p><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="comment">// r, err := consul.NewConsulRegister(&quot;localhost:8500&quot;)</span></span><br><span class="line">r, err := consul.NewConsulRegister(conf.Getconf().Registry.RegistryAddress[<span class="number">0</span>])</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">    log.Fatal(err)</span><br><span class="line">&#125;</span><br><span class="line">opts = <span class="built_in">append</span>(opts, server.WithRegistry(r))</span><br></pre></td></tr></table></figure><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">version:</span> <span class="string">&#x27;3&#x27;</span></span><br><span class="line"><span class="attr">services:</span></span><br><span class="line">  <span class="attr">consul:</span></span><br><span class="line">    <span class="attr">ports:</span></span><br><span class="line">      <span class="bullet">-</span> <span class="number">8500</span><span class="string">:8500</span></span><br></pre></td></tr></table></figure><h1 id="gorm操作数据库"><a class="markdownIt-Anchor" href="#gorm操作数据库"></a> Gorm操作数据库</h1><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> model</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> <span class="string">&quot;gorm.io/gorm&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> User <span class="keyword">struct</span> &#123;</span><br><span class="line">    gorm.Model</span><br><span class="line">    Email <span class="type">string</span> <span class="string">`gorm:&quot;uniqueIndex;type:varchar(128) not null&quot;`</span></span><br><span class="line">    Password <span class="type">string</span> <span class="string">`gorm:&quot;type:varchar(64) not null&quot;`</span></span><br><span class="line"></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h1 id="新增页面"><a class="markdownIt-Anchor" href="#新增页面"></a> 新增页面</h1><ul><li>路由</li></ul><figure class="highlight go"><table><tr><td class="code"><pre><span class="line"><span class="comment">// main.go</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    h.GET(<span class="string">&quot;/your-page&quot;</span>, <span class="function"><span class="keyword">func</span><span class="params">(c context.Context, ctx *app.RequestContext)</span></span> &#123;</span><br><span class="line">        ctx.HTML(consts.StatusOK, <span class="string">&quot;your-page.tmpl&quot;</span>, utils.H(<span class="string">&quot;Title: Your Title&quot;</span>))</span><br><span class="line">    &#125;)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><ul><li>模板</li></ul><figure class="highlight html"><table><tr><td class="code"><pre><span class="line">// your-page.tmpl</span><br><span class="line">&#123;&#123; define &quot;your-page&quot; &#125;&#125;</span><br><span class="line"><span class="tag">&lt;<span class="name">div</span>&gt;</span></span><br><span class="line">    ...</span><br><span class="line"><span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">&#123;&#123; end &#125;&#125;</span><br></pre></td></tr></table></figure><ul><li>Hertz生成IDL接口代码</li></ul><figure class="highlight proto"><table><tr><td class="code"><pre><span class="line">syntax = <span class="string">&quot;proto3&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">package</span> pbapi;</span><br><span class="line"></span><br><span class="line"><span class="keyword">option</span> go_package = <span class="string">&quot;/pbapi&quot;</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">message </span><span class="title class_">Request</span> &#123;</span><br><span class="line">    <span class="type">string</span> msg = <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">message </span><span class="title class_">Response</span> &#123;</span><br><span class="line">    <span class="type">string</span> msg = <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">service </span><span class="title class_">EchoService</span> &#123;</span><br><span class="line">    <span class="function"><span class="keyword">rpc</span> Echo (Request) <span class="keyword">returns</span> (Response) </span>&#123;&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;hello-world&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#hello-world&quot;&gt;&lt;/a&gt; Hello World&lt;/h1&gt;
&lt;figure class=&quot;highlight go&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td clas</summary>
      
    
    
    
    
    <category term="基本操作" scheme="http://simuleite.github.io/tags/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/"/>
    
  </entry>
  
  <entry>
    <title>AI专栏</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/AI%E4%B8%93%E6%A0%8F/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/AI%E4%B8%93%E6%A0%8F/</id>
    <published>2025-05-19T16:00:00.000Z</published>
    <updated>2025-05-29T03:06:41.629Z</updated>
    
    <content type="html"><![CDATA[<h1 id="mcp"><a class="markdownIt-Anchor" href="#mcp"></a> MCP</h1><h2 id="rag的局限性"><a class="markdownIt-Anchor" href="#rag的局限性"></a> RAG的局限性</h2><p>对于AI来说，RAG仅仅是外部知识库，AI只起到一个总结效果。而总结的效果取决于向量相似度匹配，可能遗漏关键信息。</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">direction: right</span><br><span class="line">结构化数据 -&gt; 文本块</span><br><span class="line">非结构化数据 -&gt; 文本块</span><br><span class="line"></span><br><span class="line">文本块 -&gt; 向量数据库 -&gt; 检索文本块 -&gt; 生成最终响应</span><br></pre></td></tr></table></figure><ul><li>生成内容不完整：RAG处理的是文本的切片，因此无法看到整篇文档信息。</li><li>RAG无法判断需要多少切片才能解决问题。</li><li>多轮检索能力弱。</li></ul><h2 id="mcp基础"><a class="markdownIt-Anchor" href="#mcp基础"></a> MCP基础</h2><h2 id="function-calling"><a class="markdownIt-Anchor" href="#function-calling"></a> Function Calling</h2><p>Coze的Agent就是基于Function Calling思路封装的。</p><blockquote><p>不过Function Calling成本比较高，需要模型经过专门训练微调才能稳定支持。</p></blockquote><p>这导致有些模型不支持某些插件的调用（例如Trae只有选择Sonnet、GPT等模型才可以处理图片）。</p><blockquote><p>另外，Function Calling不是一项标准，许多模型的实现细节不一样。</p></blockquote><h2 id="model-context-protocol"><a class="markdownIt-Anchor" href="#model-context-protocol"></a> Model Context Protocol</h2><p>MCP是一项标准<strong>协议</strong>，简单来说就是通用的接口，使AI-外部工具/数据源交互标准化、可复用。</p><p>Claude Desktop、Cursor这样的工具在内部实现MCP Client，这个Client通过MCP协议与MCP Server（由服务提供公司自己开发，实现访问数据、浏览器、本地文件等功能，最终通过MCP返回标准格式）交互，最终在MCP Host上展示。</p><h2 id="mcp-传输方式"><a class="markdownIt-Anchor" href="#mcp-传输方式"></a> MCP 传输方式</h2><p>STDIO，本地环境<br />SSE，并发量不高，单向通信<br />Streamable HTTP，高并发，需要维护长连接</p><table><thead><tr><th>指标</th><th>Function Calling</th><th>Model Context Portocol</th></tr></thead><tbody><tr><td>协议</td><td>私有协议</td><td>开放协议</td></tr><tr><td>场景</td><td>单次函数调用</td><td>多工具协同 + 数据交互</td></tr><tr><td>接入方式</td><td>函数直接接入</td><td>需要MCP Server + MCP Client</td></tr><tr><td>耦合度</td><td>工具与模型绑定</td><td>工具开发与Agent开发解耦</td></tr><tr><td>调用方式</td><td>API</td><td>Stdio/SSE</td></tr></tbody></table><span id="more"></span><h2 id="mcp五大能力"><a class="markdownIt-Anchor" href="#mcp五大能力"></a> MCP五大能力</h2><ul><li>Tools：提供功能，使LLM能与外部系统交互</li><li>Resources：提供内容和数据，为LLM和Client提供上下文</li><li>Prompts：提供Prompt模板，引导LLM交互</li><li>Sampling：Server借助Client向LLM发起完成请求，实现复杂功能。</li><li>Roots：Client为Server提供一些资源地址，使Server知道去哪里获取资源。</li></ul><h2 id="mcp-server"><a class="markdownIt-Anchor" href="#mcp-server"></a> MCP Server</h2><p>基本功能</p><ul><li>文件、数据访问。如File System MCP Server</li><li>Web自动化：操作浏览器，如Puppeteer MCP Server</li><li>三方工具集成。如高德地图MCP Server<br /><a href="https://github.com/modelcontextprotocol/servers">MCP Server集合（官方）</a><br /><a href="https://mcp.so/">MCP.so</a><br /><a href="https://mcpmarket.cn/">MCP Market</a><br />​Cherry Stdio可以自动安装MCP Server，但是Windsurf就需要手动安装。</li></ul><h2 id="场景mcp-数据库更好的rag"><a class="markdownIt-Anchor" href="#场景mcp-数据库更好的rag"></a> 场景：MCP + 数据库，更好的RAG</h2><p>可以通过MCP使LLM接入数据库，无须手动调用sql接口。</p><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">  <span class="attr">&quot;mcpServers&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">    <span class="attr">&quot;mongodb&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">      <span class="attr">&quot;command&quot;</span><span class="punctuation">:</span> <span class="string">&quot;npx&quot;</span><span class="punctuation">,</span></span><br><span class="line">      <span class="attr">&quot;args&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="string">&quot;mcp-mongo-server&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="string">&quot;mongodb://site:port/your_table?authSource=admin&quot;</span></span><br><span class="line">      <span class="punctuation">&#125;</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line">  <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br></pre></td></tr></table></figure><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">  <span class="attr">&quot;mcpServers&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">    <span class="attr">&quot;mongodb&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">      <span class="attr">&quot;command&quot;</span><span class="punctuation">:</span> <span class="string">&quot;npx&quot;</span><span class="punctuation">,</span></span><br><span class="line">      <span class="comment">// 自动安装</span></span><br><span class="line">      <span class="attr">&quot;args&quot;</span><span class="punctuation">:</span> <span class="punctuation">[</span></span><br><span class="line">        <span class="string">&quot;-y&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="string">&quot;@modelcontextprotocol/server-filesystem&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="string">&quot;~/Downloads&quot;</span></span><br><span class="line">      <span class="punctuation">]</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line">  <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br><span class="line"></span><br></pre></td></tr></table></figure><p>通过这种方式，可以比简单地将数据库内容放到知识库中实现更好的效果。</p><h2 id="缺点"><a class="markdownIt-Anchor" href="#缺点"></a> 缺点</h2><p>MCP会真的调用sql，如果让AI检索大量数据，会消耗大量token，阻塞MCP Client。</p><blockquote><p>许多MCP Client依靠大量系统提示词来实现与MCP的通信，使用MCP，token消耗一定增加。</p></blockquote><h1 id="向量数据库"><a class="markdownIt-Anchor" href="#向量数据库"></a> 向量数据库</h1><p>向量数据库专门存储和查询向量，核心就是相似度查询。<br />向量数据的特点是单条数据维度高、存储空间占用大。<br />向量检索只能做到相似查找，返回前K条数据。</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">chunk: &#123;</span><br><span class="line">    &quot;&#123;text0: content...&#125;\n&#123;text1: content...&#125;&quot;</span><br><span class="line">&#125;</span><br><span class="line">vector: &#123;</span><br><span class="line">    &quot;&#123;\n text0: content...\n vector:[0.23xx, 0.24xx]\n &#125;\n &#123;\n text1: content...\nvector:[0.25xx, 0.26xx]\n&#125;&quot;</span><br><span class="line">&#125;</span><br><span class="line">知识库文本 -&gt; chunk: split</span><br><span class="line">chunk -&gt; vector: Embedding</span><br><span class="line">chunk -&gt; 向量模型</span><br><span class="line">vector -&gt; 向量数据库: insert</span><br><span class="line"></span><br><span class="line">向量模型 -&gt; vector</span><br><span class="line">向量模型 -&gt; 向量数据库</span><br><span class="line">向量模型 -&gt; LLM: Question</span><br><span class="line"></span><br><span class="line">用户 -&gt; 向量模型: Question</span><br><span class="line">LLM -&gt; 用户: Answer</span><br></pre></td></tr></table></figure><ol><li>文本向量化（分词、分chunk，通过向量模型转化为向量）。</li><li>将向量化数据和原始文本一起存储到向量数据库。</li><li>向量模型检索向量数据库，并传参给LLM回复。</li></ol><h2 id="为什么分块"><a class="markdownIt-Anchor" href="#为什么分块"></a> 为什么分块</h2><p>首先大模型输入有限制，其次文本分块，只检索关键文本，可以减少token消耗。</p><h2 id="建立向量索引"><a class="markdownIt-Anchor" href="#建立向量索引"></a> 建立向量索引</h2><p>想要加速向量搜索速度就需要向量索引。</p><h3 id="flat索引"><a class="markdownIt-Anchor" href="#flat索引"></a> FLAT索引</h3><p>将向量以列表形式存储，不压缩和聚类。简单但是效率低。</p><h3 id="倒排文件索引无聚类"><a class="markdownIt-Anchor" href="#倒排文件索引无聚类"></a> 倒排文件索引（无聚类）</h3><p>IVF, Inverted File Index</p><ul><li>单词词典：以文档文本单词作为主键（也就是说是唯一的），每个单词关联一个或多个文档ID。</li><li>倒排列表：记录每个单词存储在哪些文档中，以及单词在文档中的位置。</li></ul><h4 id="ivf聚类索引"><a class="markdownIt-Anchor" href="#ivf聚类索引"></a> IVF聚类索引</h4><p>IVF将原始数据划分为多个簇，为每个簇建立倒排索引，从而加快检索效率。<br />IVF_FLAT：聚类倒排+Flat结构，适合高精度，性能不高。<br />PQ, Produce Quantizer，乘积量化，通过向量分割与量化<br />IVF_PQ：通过乘积量化压缩向量，加速搜索</p><h3 id="kd树索引"><a class="markdownIt-Anchor" href="#kd树索引"></a> KD树索引</h3><p>二叉树结构，存储多维向量数据。按照层级组织数据从而实现高效的向量搜索。</p><h3 id="ball树索引"><a class="markdownIt-Anchor" href="#ball树索引"></a> Ball树索引</h3><p>非平衡树结构，类似于球形。</p><h3 id="hnsw索引"><a class="markdownIt-Anchor" href="#hnsw索引"></a> HNSW索引</h3><p>Hierarchical Navigable Small World<br />图结构，通过分层图加速效率。在每一层将向量连接成图，查询时从高层图到低层图，直到找到最近邻。<br />HNSW在召回率和索引性能之间有较好的平衡。</p><h3 id="locality-sensitive-hashing"><a class="markdownIt-Anchor" href="#locality-sensitive-hashing"></a> Locality Sensitive Hashing</h3><p>LSH将相似向量映射到同一个桶来加速检索。适合高维稀疏向量数据。</p><h2 id="向量库"><a class="markdownIt-Anchor" href="#向量库"></a> 向量库</h2><p>向量库存储静态数据。向量库只存储Embeddeing向量嵌入，不存储Embedding关联对象（因此不需要修改）。</p><ul><li>FAISS</li><li>HNSWLib</li><li>ANNOY</li></ul><h2 id="相似度"><a class="markdownIt-Anchor" href="#相似度"></a> 相似度</h2><h3 id="欧几里得距离"><a class="markdownIt-Anchor" href="#欧几里得距离"></a> 欧几里得距离</h3><p>欧式算法通过<strong>距离</strong>来匹配相似度<br />欧式算法可以反映向量绝对距离，适合需要量化数据的场景。<br />例如推荐系统，需要的不仅仅是历史行为，还要历史数据的数值。</p><p>二维欧式距离<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>d</mi><mo>=</mo><msqrt><mrow><mo stretchy="false">(</mo><msub><mi>x</mi><mn>2</mn></msub><mo>−</mo><msub><mi>x</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mo>+</mo><mo stretchy="false">(</mo><msub><mi>y</mi><mn>2</mn></msub><mo>−</mo><msub><mi>y</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow></msqrt></mrow><annotation encoding="application/x-tex">d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathdefault">d</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1.24em;vertical-align:-0.30499999999999994em;"></span><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.935em;"><span class="svg-align" style="top:-3.2em;"><span class="pstrut" style="height:3.2em;"></span><span class="mord" style="padding-left:1em;"><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.03588em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.03588em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span><span style="top:-2.8950000000000005em;"><span class="pstrut" style="height:3.2em;"></span><span class="hide-tail" style="min-width:1.02em;height:1.28em;"><svg width='400em' height='1.28em' viewBox='0 0 400000 1296' preserveAspectRatio='xMinYMin slice'><path d='M263,681c0.7,0,18,39.7,52,119c34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120c340,-704.7,510.7,-1060.3,512,-1067c4.7,-7.3,11,-11,19,-11H40000v40H1012.3s-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232c-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1s-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z M1001 80H40000v40H1012z'/></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.30499999999999994em;"><span></span></span></span></span></span></span></span></span></p><p>三维<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>d</mi><mo>=</mo><msqrt><mrow><mo stretchy="false">(</mo><msub><mi>x</mi><mn>2</mn></msub><mo>−</mo><msub><mi>x</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mo>+</mo><mo stretchy="false">(</mo><msub><mi>y</mi><mn>2</mn></msub><mo>−</mo><msub><mi>y</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mo>+</mo><mo stretchy="false">(</mo><msub><mi>z</mi><mn>2</mn></msub><mo>−</mo><msub><mi>z</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow></msqrt></mrow><annotation encoding="application/x-tex">d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2 + (z_2 - z_1)^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathdefault">d</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1.24em;vertical-align:-0.30499999999999994em;"></span><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.935em;"><span class="svg-align" style="top:-3.2em;"><span class="pstrut" style="height:3.2em;"></span><span class="mord" style="padding-left:1em;"><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.03588em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.03588em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault" style="margin-right:0.04398em;">z</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.04398em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right:0.04398em;">z</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:-0.04398em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span><span style="top:-2.8950000000000005em;"><span class="pstrut" style="height:3.2em;"></span><span class="hide-tail" style="min-width:1.02em;height:1.28em;"><svg width='400em' height='1.28em' viewBox='0 0 400000 1296' preserveAspectRatio='xMinYMin slice'><path d='M263,681c0.7,0,18,39.7,52,119c34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120c340,-704.7,510.7,-1060.3,512,-1067c4.7,-7.3,11,-11,19,-11H40000v40H1012.3s-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232c-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1s-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z M1001 80H40000v40H1012z'/></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.30499999999999994em;"><span></span></span></span></span></span></span></span></span></p><p>依此类推…</p><h3 id="余弦相似度"><a class="markdownIt-Anchor" href="#余弦相似度"></a> 余弦相似度</h3><p>余弦相似度通过<strong>角度</strong>匹配相似度<br />很明显，余弦相似度只关注角度，适合高维相似度检索，如语义搜索、文档分类<br />余弦相似度范围从-1到1，1表示完全相似0度、0表示垂直90度、-1表示完全相反180度</p><p>向量点积除以向量模长<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>c</mi><mi>o</mi><mi>s</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mi>A</mi><mo>⋅</mo><mi>B</mi></mrow><mrow><mi mathvariant="normal">∣</mi><mover accent="true"><mi>A</mi><mo>⃗</mo></mover><mi mathvariant="normal">∣</mi><mo>⋅</mo><mi mathvariant="normal">∣</mi><mover accent="true"><mi>B</mi><mo>⃗</mo></mover><mi mathvariant="normal">∣</mi></mrow></mfrac></mrow><annotation encoding="application/x-tex">cos(\theta) = \frac{A \cdot B} {|\vec A| \cdot |\vec B|}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault">c</span><span class="mord mathdefault">o</span><span class="mord mathdefault">s</span><span class="mopen">(</span><span class="mord mathdefault" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1.5337619999999998em;vertical-align:-0.6614309999999999em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.872331em;"><span style="top:-2.513569em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">∣</span><span class="mord accent mtight"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.9663299999999999em;"><span style="top:-2.714em;"><span class="pstrut" style="height:2.714em;"></span><span class="mord mathdefault mtight">A</span></span><span style="top:-2.96633em;"><span class="pstrut" style="height:2.714em;"></span><span class="accent-body" style="left:-0.09660999999999997em;"><span class="overlay mtight" style="height:0.714em;width:0.471em;"><svg width='0.471em' height='0.714em' style='width:0.471em' viewBox='0 0 471 714' preserveAspectRatio='xMinYMin'><path d='M377 20c0-5.333 1.833-10 5.5-14S391 0 397 0c4.667 0 8.667 1.667 12 53.333 2.667 6.667 9 10 19 6.667 24.667 20.333 43.667 41 57 7.333 4.667 1110.667 11 18 0 6-1 10-3 12s-6.667 5-14 9c-28.667 14.667-53.667 35.667-75 63-1.333 1.333-3.167 3.5-5.5 6.5s-4 4.833-5 5.5c-1 .667-2.5 1.333-4.5 2s-4.333 1-7 1c-4.667 0-9.167-1.833-13.5-5.5S337 184 337 178c0-12.667 15.667-32.333 47-59H213l-171-1c-8.667-6-13-12.333-13-19 0-4.667 4.333-11.333 13-20h359c-16-25.333-24-45-24-59z'/></svg></span></span></span></span></span></span></span><span class="mord mtight">∣</span><span class="mbin mtight">⋅</span><span class="mord mtight">∣</span><span class="mord accent mtight"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.9663299999999999em;"><span style="top:-2.714em;"><span class="pstrut" style="height:2.714em;"></span><span class="mord mathdefault mtight" style="margin-right:0.05017em;">B</span></span><span style="top:-2.96633em;"><span class="pstrut" style="height:2.714em;"></span><span class="accent-body" style="left:-0.15216em;"><span class="overlay mtight" style="height:0.714em;width:0.471em;"><svg width='0.471em' height='0.714em' style='width:0.471em' viewBox='0 0 471 714' preserveAspectRatio='xMinYMin'><path d='M377 20c0-5.333 1.833-10 5.5-14S391 0 397 0c4.667 0 8.667 1.667 12 53.333 2.667 6.667 9 10 19 6.667 24.667 20.333 43.667 41 57 7.333 4.667 1110.667 11 18 0 6-1 10-3 12s-6.667 5-14 9c-28.667 14.667-53.667 35.667-75 63-1.333 1.333-3.167 3.5-5.5 6.5s-4 4.833-5 5.5c-1 .667-2.5 1.333-4.5 2s-4.333 1-7 1c-4.667 0-9.167-1.833-13.5-5.5S337 184 337 178c0-12.667 15.667-32.333 47-59H213l-171-1c-8.667-6-13-12.333-13-19 0-4.667 4.333-11.333 13-20h359c-16-25.333-24-45-24-59z'/></svg></span></span></span></span></span></span></span><span class="mord mtight">∣</span></span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">A</span><span class="mbin mtight">⋅</span><span class="mord mathdefault mtight" style="margin-right:0.05017em;">B</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.6614309999999999em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></p><p>高维算法<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>A</mi><mo>=</mo><mo stretchy="false">(</mo><msub><mi>a</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>a</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>a</mi><mi>n</mi></msub><mo stretchy="false">)</mo><mo separator="true">;</mo><mi>B</mi><mo>=</mo><mo stretchy="false">(</mo><msub><mi>b</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>b</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>b</mi><mi>n</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">A = (a_1, a_2, ..., a_n); B = (b_1, b_2, ..., b_n)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault">A</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mpunct">;</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord mathdefault" style="margin-right:0.05017em;">B</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></p><p><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>A</mi><mo>⋅</mo><mi>B</mi><mo>=</mo><mo stretchy="false">(</mo><msub><mi>a</mi><mn>1</mn></msub><msub><mi>b</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>a</mi><mn>2</mn></msub><msub><mi>b</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>a</mi><mi>n</mi></msub><msub><mi>b</mi><mi>n</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">A \cdot B = (a_1b_1, a_2b_2, ..., a_nb_n)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault">A</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.05017em;">B</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></p><p><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi mathvariant="normal">∣</mi><mover accent="true"><mi>A</mi><mo>⃗</mo></mover><mi mathvariant="normal">∣</mi><mo>⋅</mo><mi mathvariant="normal">∣</mi><mover accent="true"><mi>B</mi><mo>⃗</mo></mover><mi mathvariant="normal">∣</mi><mo>=</mo><msqrt><mrow><msubsup><mi>a</mi><mn>1</mn><mn>2</mn></msubsup><mo>+</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo>+</mo><msubsup><mi>a</mi><mi>n</mi><mn>2</mn></msubsup></mrow></msqrt><mo>⋅</mo><msqrt><mrow><msubsup><mi>b</mi><mn>1</mn><mn>2</mn></msubsup><mo>+</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo>+</mo><msubsup><mi>b</mi><mi>n</mi><mn>2</mn></msubsup></mrow></msqrt></mrow><annotation encoding="application/x-tex">|\vec A| \cdot |\vec B| = \sqrt{a_1^2 + ... + a_n^2} \cdot \sqrt{b_1^2 + ... + b_n^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.21633em;vertical-align:-0.25em;"></span><span class="mord">∣</span><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.9663299999999999em;"><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span class="mord mathdefault">A</span></span><span style="top:-3.25233em;"><span class="pstrut" style="height:3em;"></span><span class="accent-body" style="left:-0.09660999999999997em;"><span class="overlay" style="height:0.714em;width:0.471em;"><svg width='0.471em' height='0.714em' style='width:0.471em' viewBox='0 0 471 714' preserveAspectRatio='xMinYMin'><path d='M377 20c0-5.333 1.833-10 5.5-14S391 0 397 0c4.667 0 8.667 1.667 12 53.333 2.667 6.667 9 10 19 6.667 24.667 20.333 43.667 41 57 7.333 4.667 1110.667 11 18 0 6-1 10-3 12s-6.667 5-14 9c-28.667 14.667-53.667 35.667-75 63-1.333 1.333-3.167 3.5-5.5 6.5s-4 4.833-5 5.5c-1 .667-2.5 1.333-4.5 2s-4.333 1-7 1c-4.667 0-9.167-1.833-13.5-5.5S337 184 337 178c0-12.667 15.667-32.333 47-59H213l-171-1c-8.667-6-13-12.333-13-19 0-4.667 4.333-11.333 13-20h359c-16-25.333-24-45-24-59z'/></svg></span></span></span></span></span></span></span><span class="mord">∣</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.21633em;vertical-align:-0.25em;"></span><span class="mord">∣</span><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.9663299999999999em;"><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span class="mord mathdefault" style="margin-right:0.05017em;">B</span></span><span style="top:-3.25233em;"><span class="pstrut" style="height:3em;"></span><span class="accent-body" style="left:-0.15216em;"><span class="overlay" style="height:0.714em;width:0.471em;"><svg width='0.471em' height='0.714em' style='width:0.471em' viewBox='0 0 471 714' preserveAspectRatio='xMinYMin'><path d='M377 20c0-5.333 1.833-10 5.5-14S391 0 397 0c4.667 0 8.667 1.667 12 53.333 2.667 6.667 9 10 19 6.667 24.667 20.333 43.667 41 57 7.333 4.667 1110.667 11 18 0 6-1 10-3 12s-6.667 5-14 9c-28.667 14.667-53.667 35.667-75 63-1.333 1.333-3.167 3.5-5.5 6.5s-4 4.833-5 5.5c-1 .667-2.5 1.333-4.5 2s-4.333 1-7 1c-4.667 0-9.167-1.833-13.5-5.5S337 184 337 178c0-12.667 15.667-32.333 47-59H213l-171-1c-8.667-6-13-12.333-13-19 0-4.667 4.333-11.333 13-20h359c-16-25.333-24-45-24-59z'/></svg></span></span></span></span></span></span></span><span class="mord">∣</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1.24em;vertical-align:-0.2902em;"></span><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.9498em;"><span class="svg-align" style="top:-3.2em;"><span class="pstrut" style="height:3.2em;"></span><span class="mord" style="padding-left:1em;"><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.7959080000000001em;"><span style="top:-2.433692em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.0448000000000004em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.26630799999999993em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.4530000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.247em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.9098em;"><span class="pstrut" style="height:3.2em;"></span><span class="hide-tail" style="min-width:1.02em;height:1.28em;"><svg width='400em' height='1.28em' viewBox='0 0 400000 1296' preserveAspectRatio='xMinYMin slice'><path d='M263,681c0.7,0,18,39.7,52,119c34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120c340,-704.7,510.7,-1060.3,512,-1067c4.7,-7.3,11,-11,19,-11H40000v40H1012.3s-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232c-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1s-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z M1001 80H40000v40H1012z'/></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2902em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.24em;vertical-align:-0.2902em;"></span><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.9498em;"><span class="svg-align" style="top:-3.2em;"><span class="pstrut" style="height:3.2em;"></span><span class="mord" style="padding-left:1em;"><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.7959080000000001em;"><span style="top:-2.433692em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.0448000000000004em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.26630799999999993em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.740108em;"><span style="top:-2.4530000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span><span style="top:-2.9890000000000003em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.247em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.9098em;"><span class="pstrut" style="height:3.2em;"></span><span class="hide-tail" style="min-width:1.02em;height:1.28em;"><svg width='400em' height='1.28em' viewBox='0 0 400000 1296' preserveAspectRatio='xMinYMin slice'><path d='M263,681c0.7,0,18,39.7,52,119c34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120c340,-704.7,510.7,-1060.3,512,-1067c4.7,-7.3,11,-11,19,-11H40000v40H1012.3s-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232c-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1s-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z M1001 80H40000v40H1012z'/></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2902em;"><span></span></span></span></span></span></span></span></span></p><h3 id="点积相似度投影"><a class="markdownIt-Anchor" href="#点积相似度投影"></a> 点积相似度（投影）</h3><p>不除以模长，与角度无关；对长度敏感，两个向量长度不同，方向相同，点积会受影响。<br />点积正数，两个向量正相关；点积为0，无线性相关；点积负数，两个向量负相关<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>A</mi><mo>⋅</mo><mi>B</mi><mo>=</mo><mo stretchy="false">(</mo><msub><mi>a</mi><mn>1</mn></msub><msub><mi>b</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>a</mi><mn>2</mn></msub><msub><mi>b</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>a</mi><mi>n</mi></msub><msub><mi>b</mi><mi>n</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">A \cdot B = (a_1b_1, a_2b_2, ..., a_nb_n)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault">A</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.05017em;">B</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.30110799999999993em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord mathdefault">a</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault">b</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></p><h1 id="高并发下大模型性能"><a class="markdownIt-Anchor" href="#高并发下大模型性能"></a> 高并发下大模型性能</h1><ul><li>首包延迟：用户首次接收到模型响应的时间，对于一些思考模型来说，可能会很长。</li><li>模型并发能力：有的大模型/平台有并发数限制。</li><li>输出超限：要计算token，看模型是否一次返回的内容过多，超出限制导致无效调用。<sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup></li><li>兜底返回：调用模型有时时间太长，不确定模型有没有响应，最好设置一个超时重试时间。</li><li>多线程：使用线程池来多线程批量请求。<br />批量请求是指凑齐token为512，确保批次张量维度一致。</li></ul><h1 id="隐私计算"><a class="markdownIt-Anchor" href="#隐私计算"></a> 隐私计算</h1><h2 id="secretflow"><a class="markdownIt-Anchor" href="#secretflow"></a> SecretFlow</h2><p>举个例子，A、B、C三个人想要知道他们的平均工资，但是又不希望透露隐私，他们可以这么做：</p><h3 id="过程加密"><a class="markdownIt-Anchor" href="#过程加密"></a> 过程加密</h3><h3 id="mpc"><a class="markdownIt-Anchor" href="#mpc"></a> MPC</h3><p>Multi-Party Computation<br />MPC，多方安全计算：使用MPC三方分别加密自己的数据，其他人不能解密</p><ul><li><strong>秘密分享法（Secret Sharing）</strong>：把数据切片分发。</li><li><strong>加密协议法</strong>：用加密算法让多方协作计算。</li><li><strong>布尔电路/算数电路法</strong>：把整个计算过程拆成很多小步骤，每一步都用安全协议保护。</li></ul><h3 id="he"><a class="markdownIt-Anchor" href="#he"></a> HE</h3><p>Homomorphic Encryption<br />HE，同态加密：使用加密数据运算，但是仍然能得出相同的加密后的结果</p><p>普通加密只能保密，不能计算，但是同态加密专门设计成可以计算的加密算法，保证加密后运算的结果密文，解密后和明文计算结果一致。</p><ul><li>部分同态加密，<strong>只能</strong>加或者乘，如Paillier</li><li>全同态加密，<strong>同时</strong>支持加和乘，如Gentry方案、CKKS</li></ul><h3 id="tee"><a class="markdownIt-Anchor" href="#tee"></a> TEE</h3><p>Trusted Execution Environment<br />TEE，可信计算环境：计算过程放在一个可信的小黑盒里，不让外面看见</p><p>TEE会在OS内部抽象出一个小黑盒，黑盒以外的部分，OS无法窥视黑盒的内容。<br />数据在TEE之外是加密的，在TEE之内才会解密运算。</p><h4 id="如何保证计算结果正确"><a class="markdownIt-Anchor" href="#如何保证计算结果正确"></a> 如何保证计算结果正确</h4><p>Remote Attestation<br />RA，远程认证：TEE启动会生成一份报告，包括TEE身份指纹、当前程序Hash、运行环境信息。根据Hash可以验证是不是同一份程序。</p><h3 id="结果加密"><a class="markdownIt-Anchor" href="#结果加密"></a> 结果加密</h3><h3 id="dp"><a class="markdownIt-Anchor" href="#dp"></a> DP</h3><p>Differential Privacy<br />DP，差分隐私：在数据里随机加入噪音，让人类无法识别。在数据量庞大的时候，整体的噪音量是很小的，不影响计算结果。</p><h3 id="fl"><a class="markdownIt-Anchor" href="#fl"></a> FL</h3><p>Federated Learning<br />FL，多方想要统计一份数据，如果是把多方数据汇合到一处进行统计，那容易泄露隐私。<br />因此联邦学习是将初始模型分发给各方，各方在本地统计、训练，最终将训练成果反馈给总部。</p><hr class="footnotes-sep" /><section class="footnotes"><ol class="footnotes-list"><li id="fn1" class="footnote-item"><p>问小白就有这个问题 <a href="#fnref1" class="footnote-backref">↩︎</a></p></li></ol></section>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;mcp&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#mcp&quot;&gt;&lt;/a&gt; MCP&lt;/h1&gt;
&lt;h2 id=&quot;rag的局限性&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#rag的局限性&quot;&gt;&lt;/a&gt; RAG的局限性&lt;/h2&gt;
&lt;p&gt;对于AI来说，RAG仅仅是外部知识库，AI只起到一个总结效果。而总结的效果取决于向量相似度匹配，可能遗漏关键信息。&lt;/p&gt;
&lt;figure class=&quot;highlight plaintext&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;direction: right&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;结构化数据 -&amp;gt; 文本块&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;非结构化数据 -&amp;gt; 文本块&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;文本块 -&amp;gt; 向量数据库 -&amp;gt; 检索文本块 -&amp;gt; 生成最终响应&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;ul&gt;
&lt;li&gt;生成内容不完整：RAG处理的是文本的切片，因此无法看到整篇文档信息。&lt;/li&gt;
&lt;li&gt;RAG无法判断需要多少切片才能解决问题。&lt;/li&gt;
&lt;li&gt;多轮检索能力弱。&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;mcp基础&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#mcp基础&quot;&gt;&lt;/a&gt; MCP基础&lt;/h2&gt;
&lt;h2 id=&quot;function-calling&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#function-calling&quot;&gt;&lt;/a&gt; Function Calling&lt;/h2&gt;
&lt;p&gt;Coze的Agent就是基于Function Calling思路封装的。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;不过Function Calling成本比较高，需要模型经过专门训练微调才能稳定支持。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;这导致有些模型不支持某些插件的调用（例如Trae只有选择Sonnet、GPT等模型才可以处理图片）。&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;另外，Function Calling不是一项标准，许多模型的实现细节不一样。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;model-context-protocol&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#model-context-protocol&quot;&gt;&lt;/a&gt; Model Context Protocol&lt;/h2&gt;
&lt;p&gt;MCP是一项标准&lt;strong&gt;协议&lt;/strong&gt;，简单来说就是通用的接口，使AI-外部工具/数据源交互标准化、可复用。&lt;/p&gt;
&lt;p&gt;Claude Desktop、Cursor这样的工具在内部实现MCP Client，这个Client通过MCP协议与MCP Server（由服务提供公司自己开发，实现访问数据、浏览器、本地文件等功能，最终通过MCP返回标准格式）交互，最终在MCP Host上展示。&lt;/p&gt;
&lt;h2 id=&quot;mcp-传输方式&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#mcp-传输方式&quot;&gt;&lt;/a&gt; MCP 传输方式&lt;/h2&gt;
&lt;p&gt;STDIO，本地环境&lt;br /&gt;
SSE，并发量不高，单向通信&lt;br /&gt;
Streamable HTTP，高并发，需要维护长连接&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;指标&lt;/th&gt;
&lt;th&gt;Function Calling&lt;/th&gt;
&lt;th&gt;Model Context Portocol&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;协议&lt;/td&gt;
&lt;td&gt;私有协议&lt;/td&gt;
&lt;td&gt;开放协议&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;场景&lt;/td&gt;
&lt;td&gt;单次函数调用&lt;/td&gt;
&lt;td&gt;多工具协同 + 数据交互&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;接入方式&lt;/td&gt;
&lt;td&gt;函数直接接入&lt;/td&gt;
&lt;td&gt;需要MCP Server + MCP Client&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;耦合度&lt;/td&gt;
&lt;td&gt;工具与模型绑定&lt;/td&gt;
&lt;td&gt;工具开发与Agent开发解耦&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;调用方式&lt;/td&gt;
&lt;td&gt;API&lt;/td&gt;
&lt;td&gt;Stdio/SSE&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;</summary>
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>MQ消息队列</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/MQ%E6%B6%88%E6%81%AF%E9%98%9F%E5%88%97/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/MQ%E6%B6%88%E6%81%AF%E9%98%9F%E5%88%97/</id>
    <published>2025-05-11T16:00:00.000Z</published>
    <updated>2025-08-06T14:15:06.326Z</updated>
    
    <content type="html"><![CDATA[<h1 id="消息队列3大目标"><a class="markdownIt-Anchor" href="#消息队列3大目标"></a> 消息队列3大目标</h1><h2 id="异步"><a class="markdownIt-Anchor" href="#异步"></a> 异步</h2><p>在生产者-消费者速度不匹配的情况下，使用异步可以减少等待，提高效率。</p><h2 id="解耦"><a class="markdownIt-Anchor" href="#解耦"></a> 解耦</h2><p>多个生产者可以通过消息队列管道集合成1条链路；也可以将1个生产者的消息负载均衡给多个消费者（只发送1条消息给MQ，MQ广播多份）。例如，增加了一个数据分析业务，这时候不需要修改业务代码，只需要配置MQ发送相应消息到大数据系统Server即可。<br />同时，生产者只需要关心将消息发送给MQ，无需关心后续处理（消费者挂了怎么办）；MQ会负责和消费者通信。</p><h2 id="削峰生产者-消费者速度不同步"><a class="markdownIt-Anchor" href="#削峰生产者-消费者速度不同步"></a> 削峰（生产者-消费者速度不同步）</h2><p>由于队列本身是一条管道，拥有一定容量，因此可以削峰填谷，解决一些瞬时高并发流量。</p><h1 id="消息队列的关键问题"><a class="markdownIt-Anchor" href="#消息队列的关键问题"></a> 消息队列的关键问题</h1><h2 id="c-系统一致性"><a class="markdownIt-Anchor" href="#c-系统一致性"></a> C 系统一致性</h2><p>A系统通过MQ将消息发送给B、C完成后续业务，B成功而C失败，这时如何保证一致性？</p><h2 id="a-系统可用性"><a class="markdownIt-Anchor" href="#a-系统可用性"></a> A 系统可用性</h2><p>MQ宕机，依赖MQ管道的服务就不可用。MQ应该有高可用性和稳定性，不应该成为系统薄弱环节。<br />因此需要MQ集群，这时候又需要新的中间层NameSrv来管理维护MQ集群。</p><h2 id="系统复杂度"><a class="markdownIt-Anchor" href="#系统复杂度"></a> 系统复杂度</h2><ul><li>如何保证消费不丢失？</li><li>如何避免重复消费？</li><li>如何保证消息顺序？</li></ul><h2 id="幂等性"><a class="markdownIt-Anchor" href="#幂等性"></a> 幂等性</h2><blockquote><p>多次消费结果相当于只消费一次。</p></blockquote><p>可以用业务id作为消息key，对key校验有没有消费过。<br />如果重复消费，确保多次消费和1次消费的结果相同。</p><ul><li>发送消息重复：发送后，网络断开，没收到ACK，导致重复发送</li><li>消费消息重复：Consumer收到消息并处理完成，但是由于网络问题，Consumer应答没有发送到Broker；Broker遵从<strong>至少消费一次原则</strong>，重新发送。</li><li>Rebalance消息重复：Consumer Group的Consumer数量发生变化，触发Rebalance，此时Consumer可能会收到曾经被消费过的消息。</li></ul><h1 id="message-queue产品"><a class="markdownIt-Anchor" href="#message-queue产品"></a> Message Queue产品</h1><table><thead><tr><th>产品</th><th>优势</th><th>劣势</th><th>场景</th></tr></thead><tbody><tr><td>Kafaka</td><td>吞吐量大、性能高、集群高可用</td><td>丢数据、功能单一</td><td>MapReduce大数据采集、日志分析</td></tr><tr><td>RabbitMQ</td><td>消息可靠、功能全面</td><td>erlang语言不容易定制，吞吐量较低</td><td>小规模服务调用</td></tr><tr><td>Pulsar</td><td>Bookeeper，消息可靠性高</td><td>使用较少、生态有差距</td><td>大规模服务调用</td></tr><tr><td>RocketMQ</td><td>高吞吐、高性能、高可用。Java语言容易定制。</td><td>Java服务加载慢</td><td>功能全面，尤其适合金融、电商、互联网场景</td></tr></tbody></table><h1 id="消息队列工作方式"><a class="markdownIt-Anchor" href="#消息队列工作方式"></a> 消息队列工作方式</h1><p>RocketMQ和Kafka都使用Topic，每个Topic的内容会分发到多个管道（Partition或MessageQueue）。而Kafka在Topic过多的情况下，吞吐量会严重下降；RocketMQ解决了这个问题。</p><h1 id="rocketmq集群"><a class="markdownIt-Anchor" href="#rocketmq集群"></a> RocketMQ集群</h1><p>在RocketMQ集群中，多台NameSrv是平等的，而Broker会组成多个主-从结构。<br />Slave只负责备份，只有Master(brokerId=0)才会发送消息。<br />然而主从结构的Slave，由于brokerId不为0，不会自动切换为Master，需要人工介入。</p><h2 id="dledger高可用集群"><a class="markdownIt-Anchor" href="#dledger高可用集群"></a> Dledger高可用集群</h2><p>Dleger是一种Raft算法，实现了Leader选举。<br />Dledger会从Followers中自动选举Leader，从而保证高可用。</p><h1 id="三种发送方式"><a class="markdownIt-Anchor" href="#三种发送方式"></a> 三种发送方式</h1><h2 id="单向发送"><a class="markdownIt-Anchor" href="#单向发送"></a> 单向发送</h2><p>Producer只发送消息、不处理ACK；MQ也不发送ACK。消息可靠性没有保障。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 返回值为null，不处理ACK。</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">sendOneWay</span><span class="params">(Message msg)</span> <span class="keyword">throws</span> ...Exception &#123;</span><br><span class="line">    msg.setTopic(withNamespace(msg.getTopic()));</span><br><span class="line">    <span class="built_in">this</span>.defaultMQProducerImpl.sendOneWay(msg);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="同步发送"><a class="markdownIt-Anchor" href="#同步发送"></a> 同步发送</h2><p>Producer等待MQ ACK，才继续操作。同步发送可能会发生阻塞。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> SendResult <span class="title function_">sendResult</span><span class="params">(</span></span><br><span class="line"><span class="params">        Collection&lt;Message&gt; msgs)</span> <span class="keyword">throws</span> ...Exception &#123;</span><br><span class="line">    <span class="keyword">return</span> <span class="built_in">this</span>.defaultMQProducerImpl.send(batch(msgs));</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="异步发送"><a class="markdownIt-Anchor" href="#异步发送"></a> 异步发送</h2><p>Producer不等待MQ ACK（异步ACK，也能保证不丢失消息），直接发送消息。<br />但是异步发送也有代价，我们不能发送完立刻<code>producer.shutdown()</code>，而需要设置一段延迟，使producer能够捕捉Exception并重发消息。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// send方法本身没有返回值，不会阻塞；但是能够处理Exception</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">send</span><span class="params">(Message msg, </span></span><br><span class="line"><span class="params">                 SendCallBack sendCallBack)</span> <span class="keyword">throws</span> ...Exception &#123;</span><br><span class="line">    msg.setTopic(withNamespace(msg.getTopic()));</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        <span class="keyword">if</span> (<span class="built_in">this</span>.getAutoBatch() &amp;&amp; !(msg <span class="keyword">instanceof</span> MessageBatch)) &#123;</span><br><span class="line">            sendByAccumulator(msg, <span class="literal">null</span>, sendCallBack);</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            sendDirect(msg, <span class="literal">null</span>, sendCallBack);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125; <span class="keyword">catch</span> (Throwable e) &#123;</span><br><span class="line">        sendCallBack.onException(e);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">producer.send(msg, <span class="keyword">new</span> <span class="title class_">SendCallBack</span>() &#123;</span><br><span class="line">    <span class="meta">@Override</span></span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onSuccess</span><span class="params">(SendResult sendResult)</span> &#123;</span><br><span class="line">        ...</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="meta">@Override</span></span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">onException</span><span class="params">(Throwable e)</span> &#123;</span><br><span class="line">        ...</span><br><span class="line">    &#125;</span><br><span class="line">&#125;);</span><br></pre></td></tr></table></figure><h1 id="两种消费方式"><a class="markdownIt-Anchor" href="#两种消费方式"></a> 两种消费方式</h1><h2 id="consumer拉取"><a class="markdownIt-Anchor" href="#consumer拉取"></a> Consumer拉取</h2><p>Consumer维护一个轮询拉取，Broker收到拉取请求后发送消息。</p><h2 id="broker推送"><a class="markdownIt-Anchor" href="#broker推送"></a> Broker推送</h2><p><strong>一般只用推模式</strong>，因为Consumer需要轮询（即使Broker不一定有消息），会消耗部分资源。</p><h1 id="消息类型"><a class="markdownIt-Anchor" href="#消息类型"></a> 消息类型</h1><h2 id="顺序消息"><a class="markdownIt-Anchor" href="#顺序消息"></a> 顺序消息</h2><p>局部有序，实际上是序号相同的消息发送到同一个队列管道，然后消费者从一个管道中拿消息，从而保证有序性。</p><h2 id="广播消息"><a class="markdownIt-Anchor" href="#广播消息"></a> 广播消息</h2><p>正常情况下，多个Consumer是负载均衡模式，一条消息只会发到其中一个Consumer消费；而在广播模式下，所有的Consumer都会收到消息。<br />在代码层面，正常情况下<strong>服务端统一</strong>维护消费者位点；而在广播模式下<strong>客户端本地</strong><code>.rocket_offsets</code>维护消费者位点</p><h1 id="消息重试"><a class="markdownIt-Anchor" href="#消息重试"></a> 消息重试</h1><h2 id="顺序消息-2"><a class="markdownIt-Anchor" href="#顺序消息-2"></a> 顺序消息</h2><p>顺序消息要拿到ACK才会发送下一条消息，否则会重发消息</p><h2 id="无序消息"><a class="markdownIt-Anchor" href="#无序消息"></a> 无序消息</h2><p>为了保障无需消息的消费，MQ设置了一个消息重试间隔时间。如果没有回复，间隔10s-30s-1m-2m…来重发消息，最多重试16次（默认）。<br />如果达到重试上限还未消费，该消息称为<strong>死信消息</strong>。死信消息会进入<strong>死信队列</strong>。</p><h3 id="死信队列"><a class="markdownIt-Anchor" href="#死信队列"></a> 死信队列</h3><p>死信队列不归属于Topic、Consumer，而是归属于Group Id。<br />死信队列的消息不会被再次重复消费，有效期为3天，过期删除。<br />可以手工在监控平台里处理死信，获取messageId后自己处理。</p><h2 id="重复消费"><a class="markdownIt-Anchor" href="#重复消费"></a> 重复消费</h2><p>网络闪断（成功执行，MQ没收到ACK）、生产者宕机（成功发送到MQ，生产者没收到ACK）会引发重复消费。</p><h1 id="什么是消息事务"><a class="markdownIt-Anchor" href="#什么是消息事务"></a> 什么是消息事务</h1><p>消息事务基于消息队列的两阶段提交，将本地事务和发放消息放在了一个分布式事务里。保证原子性。<br />用法：将一个分布式事务拆分成一个消息事务（A系统本地操作+发消息）+ B系统本地操作。<br />B系统操作由消息驱动。只要消息事务成功，那么A操作一定成功；这时B系统收到消息执行本地操作，如果本地操作失败，消息会重新投放，直到B操作成功。</p><p>上面的方法满足BASE：B基本A可用；S软状态；E最终一致性<br />BASE是对于CAP中的AP系统的拓展。牺牲强一致性来保证Available和Performance。<br />满足BASE的事务称为“柔性事务”</p><h2 id="什么是exactly-once"><a class="markdownIt-Anchor" href="#什么是exactly-once"></a> 什么是Exactly Once</h2><h2 id="at-least-once"><a class="markdownIt-Anchor" href="#at-least-once"></a> At Least Once</h2><p>Producer接受Broker ACK来确保信息成功写入Topic。<br />如果Producer接收ACK超时、或Broker出错时，会重复发送消息。</p><p>但是如果Broker已经写入Topic，但是没有来得及发送ACK或ACK超时，Producer重新发送的消息会第二次写入Topic，导致最终Consumer收到重复消息。</p><h2 id="at-most-once"><a class="markdownIt-Anchor" href="#at-most-once"></a> At Most Once</h2><p>Producer接收ACK超时，或Broker出错时没有重复发消息，会导致消息丢失，没有写入Topic，也没有被Consumer消费。<br />有些时候我们为了避免重复消费，允许这种情况发生。</p><h2 id="exactly-once"><a class="markdownIt-Anchor" href="#exactly-once"></a> Exactly Once</h2><p>Exactly Once是说，即使重复发送了消息，Consumer只消费一次。需要消息队列Serv、Producer、Consumer协同才能实现。</p><h2 id="rocketmq事务消息"><a class="markdownIt-Anchor" href="#rocketmq事务消息"></a> RocketMQ事务消息</h2><ol><li>MQ开启一个事务Topic</li><li>事务中第一个执行的服务发送1.5条消息（0.5是因为，这条消息在事务提交前，对Consumer不可见）</li><li>1.5发送成功后，发送0.5消息的服务开始本地事务；并决定事务提交/回滚。<br />RocketMQ保证最终一致性</li></ol><h3 id="如何做到写入消息但是对用户不可见呢"><a class="markdownIt-Anchor" href="#如何做到写入消息但是对用户不可见呢"></a> 如何做到写入消息但是对用户不可见呢？</h3><p>0.5消息，备份原消息Topic和MQ，然后改变Topic为HALF_TOPIC，由于Consumer没有订阅这个Topic，所以无法消费。<br />然后RocketMQ开始定时任务，从HALF_TOPIC中拉取消息消费，并决定提交事务还是回滚。</p><h2 id="kafka幂等"><a class="markdownIt-Anchor" href="#kafka幂等"></a> Kafka幂等</h2><p>Kafka不确定是否成功发送，就一直重试，Broker保证只消费一次。</p><h3 id="幂等producer"><a class="markdownIt-Anchor" href="#幂等producer"></a> 幂等Producer</h3><p>Kafka为了保证幂等性，引入ProducerID和SequenceNumber。<br /><code>new_seq = old_seq+1: 正常消息； new_seq &lt;= old_seq : 重复消息； new_seq &gt; old_seq+1: 消息丢失；</code></p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;消息队列3大目标&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#消息队列3大目标&quot;&gt;&lt;/a&gt; 消息队列3大目标&lt;/h1&gt;
&lt;h2 id=&quot;异步&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#异步&quot;&gt;&lt;/a&gt; </summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>Advanced Java</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Advanced%20Java/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Advanced%20Java/</id>
    <published>2025-04-27T16:00:00.000Z</published>
    <updated>2025-05-22T07:39:00.056Z</updated>
    
    <content type="html"><![CDATA[<h1 id="代理"><a class="markdownIt-Anchor" href="#代理"></a> 代理</h1><blockquote><p>代理对象通过<code>invoke</code>，实现类与非核心功能的解耦。</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">main</span><span class="params">(String[] args)</span> &#123;</span><br><span class="line">    <span class="type">Payment</span> <span class="variable">payment</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Payment</span>(<span class="string">&quot;AliPay&quot;</span>);</span><br><span class="line">    <span class="type">Pay</span> <span class="variable">proxy</span> <span class="operator">=</span> ProxyUtil.createProxy(payment);</span><br><span class="line"></span><br><span class="line">    proxy.pay(amount);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Payment</span> <span class="keyword">implements</span> <span class="title class_">Pay</span> &#123;</span><br><span class="line"></span><br><span class="line">    <span class="meta">@Overide</span></span><br><span class="line">    <span class="keyword">public</span> payResp <span class="title function_">pay</span><span class="params">(BigDecimal payAmount)</span> &#123;</span><br><span class="line">        <span class="comment">// Payment Business...</span></span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">interface</span> <span class="title class_">Pay</span> &#123;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">abstract</span> payResp <span class="title function_">pay</span><span class="params">(BigDecimal payAmount)</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">ProxyUtils</span> &#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">static</span> Pay <span class="title function_">createProxy</span><span class="params">(Payment payment)</span> &#123;</span><br><span class="line">        <span class="keyword">return</span> (Pay) Proxy.newProxyInstance(</span><br><span class="line">            ProxyUtil.class.getClassLoader(),</span><br><span class="line">            <span class="keyword">new</span> <span class="title class_">Class</span>[]&#123; Pay.class &#125;,</span><br><span class="line">            <span class="keyword">new</span> <span class="title class_">InvocationHandler</span>() &#123;</span><br><span class="line"></span><br><span class="line">                <span class="meta">@Overide</span></span><br><span class="line">                <span class="keyword">public</span> Object <span class="title function_">invoke</span><span class="params">(Object proxy, Method method, Object[] args)</span> <span class="keyword">throws</span> Throwable &#123;</span><br><span class="line">                    <span class="comment">// Payment Environment Init...</span></span><br><span class="line">                    <span class="comment">// Payment Safe Guard...</span></span><br><span class="line">                    <span class="keyword">return</span> method.invoke(payment, args);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        );</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h1 id="反射"><a class="markdownIt-Anchor" href="#反射"></a> 反射</h1><blockquote><p>反射允许对成员变量、成员方法、构造方法信息进行编程访问。</p></blockquote><p>例如，IDE的智能补全、参数提示，就是使用反射实现。</p><h2 id="class字节码获取"><a class="markdownIt-Anchor" href="#class字节码获取"></a> Class字节码获取</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Class clazz;</span><br><span class="line">clazz = Class.forName(<span class="string">&quot;com.yourpackage.TargetClass&quot;</span>);</span><br><span class="line"><span class="comment">// 参数</span></span><br><span class="line">clazz = TargetClass.class;</span><br><span class="line"><span class="comment">// 有实例时</span></span><br><span class="line">clazz = instance.getClass();</span><br></pre></td></tr></table></figure><h2 id="场景"><a class="markdownIt-Anchor" href="#场景"></a> 场景</h2><p>反射可以与配置文件结合，从而动态地创建对象。例如<code>application.yml</code>里数据库的配置、端口号等。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Properties</span> <span class="variable">prop</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Properties</span>();</span><br><span class="line"><span class="type">FileInputStream</span> <span class="variable">fis</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">FileInputStream</span>(ROOT + <span class="string">&quot;src/main/resources/application.properties&quot;</span>);</span><br><span class="line"><span class="type">String</span> <span class="variable">dataSourceUrl</span> <span class="operator">=</span> (String) perp.get(<span class="string">&quot;dataSource&quot;</span>);</span><br><span class="line"><span class="type">String</span> <span class="variable">dbName</span> <span class="operator">=</span> (String) extractDbName(dataSourceUrl);</span><br><span class="line"></span><br><span class="line"><span class="type">Class</span> <span class="variable">clazz</span> <span class="operator">=</span> Class.forName(dbName);</span><br><span class="line"><span class="type">Constructor</span> <span class="variable">con</span> <span class="operator">=</span> clazz.getDeclaredConstructor();</span><br><span class="line"><span class="type">Object</span> <span class="variable">o</span> <span class="operator">=</span> con.newInstance();</span><br><span class="line"></span><br><span class="line">...</span><br></pre></td></tr></table></figure><h1 id="hashmap"><a class="markdownIt-Anchor" href="#hashmap"></a> HashMap</h1><h2 id="jdk17"><a class="markdownIt-Anchor" href="#jdk17"></a> jdk1.7</h2><blockquote><p>jdk1.7的HashMap数据结构是：数组 + 单向链表</p></blockquote><p>当哈希后，得到的数组槽位已经存放了其他元素，这时候就需要运用指针在同一个槽位存放多个元素。</p><h2 id="头插法"><a class="markdownIt-Anchor" href="#头插法"></a> 头插法</h2><p>jdk1.7使用的方法是头插法，这样就不需要遍历到链表尾部再插入，性能高。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">void</span> <span class="title function_">createEntry</span><span class="params">(<span class="type">int</span> hash, K key, V value, <span class="type">int</span> bucketIdx)</span> &#123;</span><br><span class="line">    Entry&lt;K, V&gt; e = table[bucketIdx];</span><br><span class="line">    <span class="comment">// 这里使用头插法，在槽位头部插入新元素，并指向e，成为新的槽位引用</span></span><br><span class="line">    table[bucketIdx] = <span class="keyword">new</span> <span class="title class_">Entry</span>&lt;&gt;(hash, key, value, e); </span><br><span class="line">    size++;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">Entry</span><span class="params">(<span class="type">int</span> h, K k, V v, Entry&lt;K, V&gt; n)</span> &#123;</span><br><span class="line">    <span class="built_in">this</span>.value = v;</span><br><span class="line">    <span class="built_in">this</span>.next = n;</span><br><span class="line">    <span class="built_in">this</span>.key = k;</span><br><span class="line">    <span class="built_in">this</span>.hash = h;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>这种方法需要最终更新槽位指向新插入的节点，否则单向链表找不到新插入的元素。</p><h2 id="利用2次方机器特性"><a class="markdownIt-Anchor" href="#利用2次方机器特性"></a> 利用2次方机器特性</h2><span id="more"></span><blockquote><p>HashMap实际初始化时，不是根据用户传入的容量，而是向上取整2的次方。</p></blockquote><p>这是因为2次方可以将<strong>取模</strong>优化为<strong>位运算</strong>，避免除法（在机器中是暴力试除）<br />2的幂次取模小技巧：<span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>X</mi><mspace></mspace><mspace width="0.6666666666666666em"/><mrow><mi mathvariant="normal">m</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">d</mi></mrow><mtext> </mtext><mtext> </mtext><msup><mn>2</mn><mi>n</mi></msup><mo>=</mo><mi>X</mi><mtext> &amp; </mtext><mo stretchy="false">(</mo><msup><mn>2</mn><mi>n</mi></msup><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">X \mod 2^n = X \text{ \&amp; } (2^n - 1)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathdefault" style="margin-right:0.07847em;">X</span><span class="mspace allowbreak"></span><span class="mspace" style="margin-right:0.6666666666666666em;"></span></span><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord"><span class="mord"><span class="mord mathrm">m</span><span class="mord mathrm">o</span><span class="mord mathrm">d</span></span></span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord"><span class="mord">2</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.664392em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.07847em;">X</span><span class="mord text"><span class="mord"> &amp; </span></span><span class="mopen">(</span><span class="mord"><span class="mord">2</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.664392em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathdefault mtight">n</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span></span></span></span></p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="type">int</span> <span class="title function_">indexFor</span><span class="params">(<span class="type">int</span> h, <span class="type">int</span> length)</span> &#123;</span><br><span class="line">    <span class="keyword">return</span> h &amp; (length-<span class="number">1</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">// 以2^4 = 16为例</span><br><span class="line">    0101 ****</span><br><span class="line">mod 0001 0000</span><br><span class="line">// 优化为位运算</span><br><span class="line">    0101 ****</span><br><span class="line">&amp;   0000 1111</span><br><span class="line">// ****与操作后，一定小于16</span><br></pre></td></tr></table></figure><p>位运算取最高位算法，翻倍<code>(number - 1) &lt;&lt; 1</code>后取高位1</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">roundUpToPowerOf2</span><span class="params">(<span class="type">int</span> number)</span> &#123;</span><br><span class="line">    <span class="keyword">return</span> number &gt;= MAXIMUM_CAPACITY</span><br><span class="line">            ? MAXIUM_CAPACITY</span><br><span class="line">            : (number &gt; <span class="number">1</span>) ? Integer.highestOneBit((number - <span class="number">1</span>) &lt;&lt; <span class="number">1</span>) : <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">highestOneBit</span><span class="params">(<span class="type">int</span> i)</span> &#123;</span><br><span class="line">    i |= (i &gt;&gt;  <span class="number">1</span>);</span><br><span class="line">    i |= (i &gt;&gt;  <span class="number">2</span>);</span><br><span class="line">    i |= (i &gt;&gt;  <span class="number">4</span>);</span><br><span class="line">    i |= (i &gt;&gt;  <span class="number">8</span>);</span><br><span class="line">    i |= (i &gt;&gt; <span class="number">16</span>);</span><br><span class="line">    <span class="keyword">return</span> i - (i &gt;&gt; <span class="number">1</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p><code>highestOneBit()</code>位运算原理</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">// 任意一个数</span><br><span class="line">  0001 ****</span><br><span class="line">// i |= (i &gt;&gt;  1);</span><br><span class="line">  0001 ****</span><br><span class="line">| 0000 1***</span><br><span class="line">= 0001 1***</span><br><span class="line">// i |= (i &gt;&gt;  2);</span><br><span class="line">  0001 1***</span><br><span class="line">| 0000 011*</span><br><span class="line">= 0001 111*</span><br><span class="line">// i |= (i &gt;&gt;  4);</span><br><span class="line">  0001 111*</span><br><span class="line">| 0000 0001</span><br><span class="line">= 0001 1111</span><br><span class="line">// 至此，*全部变为1</span><br><span class="line">// i |= (i &gt;&gt;  8);</span><br><span class="line">// i |= (i &gt;&gt; 16); </span><br><span class="line">// int类型最多4Byte = 32bit，此时原数i最大已经覆盖了16位1，右移即可全部覆盖</span><br><span class="line"></span><br><span class="line">// return i - (i &gt;&gt; 1);</span><br><span class="line">  0001 1111</span><br><span class="line">- 0000 1111</span><br><span class="line">= 0001 0000</span><br><span class="line">// 得到最高位的1</span><br></pre></td></tr></table></figure><h2 id="搅动算法扩散哈希差异"><a class="markdownIt-Anchor" href="#搅动算法扩散哈希差异"></a> 搅动算法扩散哈希差异</h2><blockquote><p>实际上，HashMap除了调用<code>hashCode()</code>方法以外，还会使用位运算进行搅动，达到均匀分布的效果。</p></blockquote><p>jdk1.7通过​<strong>​位运算混合哈希码的高低位信息​</strong>​，从而减少哈希冲突。<br />（如<code>0x10000000</code>和<code>0x20000000</code>），扰动后差异会扩散到更多低位（<code>0x12000000</code>和<code>0x24000000</code>），避免仅依赖低位导致冲突。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="type">int</span> <span class="title function_">hash</span><span class="params">(Object k)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">h</span> <span class="operator">=</span> hashSeed;</span><br><span class="line">    <span class="keyword">if</span> (<span class="number">0</span> != h &amp;&amp; k instanceOf String) &#123;</span><br><span class="line">        <span class="keyword">return</span> stringHash32((String) k);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    h ^= k.hashCode();</span><br><span class="line">    <span class="comment">// 确保“仅在每个比特位存在常数倍差异”的哈希码，其冲突次数有上限（默认负载因子下约8次）​。</span></span><br><span class="line">    <span class="comment">// &#x27;&gt;&gt;&gt;&#x27; 是无符号右移</span></span><br><span class="line">    h ^= (h &gt;&gt;&gt; <span class="number">20</span>) ^ (h &gt;&gt;&gt; <span class="number">12</span>);</span><br><span class="line">    <span class="keyword">return</span> h ^ (h &gt;&gt;&gt; <span class="number">7</span>) ^ (h &gt;&gt;&gt; <span class="number">4</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>jdk1.8<a href="###%E6%9B%B4%E7%AE%80%E6%B4%81%E7%9A%84%E6%90%85%E5%8A%A8%E7%AE%97%E6%B3%95">优化</a>了这里的搅动逻辑<code>(h = key.hashCode()) ^ (h &gt;&gt;&gt; 16)</code>，保证搅动足够均匀的情况下减少运算，并结合红黑树进行优化。</p><h2 id="resize重新哈希"><a class="markdownIt-Anchor" href="#resize重新哈希"></a> resize重新哈希</h2><p>resize直接*2，扩容成原来的两倍。由于初始化就设置capacity为2的幂次，所有扩容后仍然为2的幂次。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">void</span> <span class="title function_">transfer</span><span class="params">(Entry[] newTable, <span class="type">boolean</span> rehash)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">newCapacity</span> <span class="operator">=</span> newTable.length;</span><br><span class="line">    <span class="keyword">for</span> (Entry&lt;K, V&gt; e: table) &#123;</span><br><span class="line">        <span class="keyword">while</span> (<span class="literal">null</span> != e) &#123;</span><br><span class="line">            Entry&lt;K, V&gt; next = e.next;</span><br><span class="line">            <span class="keyword">if</span> (rehash) &#123;</span><br><span class="line">                <span class="comment">// 基于新capacity，重新哈希</span></span><br><span class="line">                e.hash = (<span class="literal">null</span> == e.key ? <span class="number">0</span> : hash(e.key));</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> indexFor(e.hash, newCapacity);</span><br><span class="line">            <span class="comment">// 头插法，e对应对象的next指向到newTable</span></span><br><span class="line">            e.next = newTable[i];</span><br><span class="line">            <span class="comment">// newTable[i]这个槽位指向e对应对象，保证能遍历到新插入对象</span></span><br><span class="line">            newTable[i] = e;</span><br><span class="line">            e = e.next;</span><br><span class="line">        &#125;   </span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p><code>transfer()</code>仍然使用头插法，这样会调转原来的顺序</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">// 原来可能是这样的</span><br><span class="line">e -&gt; 1</span><br><span class="line">1 -&gt; 2 -&gt; 3 -&gt; null</span><br><span class="line"></span><br><span class="line">// 头插法, e.next = newTable[i];</span><br><span class="line">1 -&gt; null(newTable[i]初始化的Entry默认为null)</span><br><span class="line">// 调整i指针, newTable[i] = e;</span><br><span class="line">i -&gt; 1</span><br><span class="line">e -&gt; 2</span><br><span class="line"></span><br><span class="line">// 头插法</span><br><span class="line">// 此时 1 -&gt; null, 头部插入 2 -&gt; 1</span><br><span class="line">2 -&gt; 1 -&gt; null</span><br><span class="line">i -&gt; 2</span><br><span class="line">e -&gt; 3</span><br><span class="line">...</span><br><span class="line"></span><br><span class="line">// 最终结果</span><br><span class="line">3 -&gt; 2 -&gt; 1 -&gt; null</span><br></pre></td></tr></table></figure><blockquote><p>从<code>transfer()</code>核心的2行代码可以看出，HashMap扩容时操作不原子，并发执行有问题。</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">Thread1: e.next = newTable[i]; <span class="comment">// old</span></span><br><span class="line">Thread2: e.next = newTable[i]; <span class="comment">// old</span></span><br><span class="line"></span><br><span class="line">Thread2: newTable[i] = e; <span class="comment">// i -&gt; 1 -&gt; old</span></span><br><span class="line">Thread1: newTable[i] = e; <span class="comment">// i -&gt; 2 -&gt; old</span></span><br></pre></td></tr></table></figure><blockquote><p>更严重的问题是，并发<code>transfer()</code>会出现循环链表</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">// 原来的链表</span></span><br><span class="line"><span class="number">1</span> -&gt; <span class="number">2</span> -&gt; <span class="number">3</span> -&gt; <span class="literal">null</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// 并发resize()</span></span><br><span class="line"><span class="comment">// Thread1完成resize()</span></span><br><span class="line"><span class="number">3</span> -&gt; <span class="number">2</span> -&gt; <span class="number">1</span> -&gt; <span class="literal">null</span></span><br><span class="line"><span class="comment">// 这时Thread2刚开始transfer，但是拿到的是旧链表 1 -&gt; 2 -&gt; 3</span></span><br><span class="line"><span class="comment">// 从1开始</span></span><br><span class="line">i -&gt; <span class="number">1</span> -&gt; old <span class="comment">// 实际上old已经变成3 -&gt; 2 -&gt; 1了</span></span><br><span class="line"><span class="comment">// 死循环！</span></span><br><span class="line">i -&gt; <span class="number">1</span> -&gt; <span class="number">3</span> -&gt; <span class="number">2</span> -&gt; <span class="number">1</span> -&gt; <span class="number">3</span> ...</span><br></pre></td></tr></table></figure><h4 id="rehash规律"><a class="markdownIt-Anchor" href="#rehash规律"></a> rehash规律</h4><p>在rehash里有一个规律：由于扩容后还是2的幂次，因此rehash的结果要么和原来相同；要么是原来的位置+高位</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">// 假设原来capacity是16，hash后使用indexFor取模</span><br><span class="line">// 原来的slot</span><br><span class="line">  0101 0101</span><br><span class="line">&amp; 0000 1111</span><br><span class="line">= 0000 0101</span><br><span class="line">// capacity &gt;&gt; 1，变成32后</span><br><span class="line">  0101 0101</span><br><span class="line">&amp; 0001 1111</span><br><span class="line">= 0001 0101</span><br><span class="line">// 取模结果是原来加上高位16；</span><br><span class="line"></span><br><span class="line">// 如果原数在高位不为1，那rehash结果和原来一致</span><br><span class="line">  0110 0101</span><br><span class="line">&amp; 0000 1111  </span><br><span class="line">&amp; 0001 1111</span><br><span class="line">= 0000 0101</span><br></pre></td></tr></table></figure><h2 id="jdk18"><a class="markdownIt-Anchor" href="#jdk18"></a> jdk1.8</h2><blockquote><p>jdk1.8的HashMap数据结构是：数组 + (单向+双向)两种链表 + 红黑树</p></blockquote><p>说用到双向链表是因为红黑树会记录父节点和子节点，相当于双向的。<br />当数组长度达到64且哈希冲突使链表长度达到8，<strong>该槽位</strong>会改用红黑树结构。<br />如果数组未达到64，只是链表长度达到8，那么会扩容。</p><h3 id="拓展为什么是8和64"><a class="markdownIt-Anchor" href="#拓展为什么是8和64"></a> 拓展：为什么是8和64</h3><ul><li>红黑树效率高，但是数组大小小于64的时候，红黑树频繁使用平衡算法开销比较大，而且红黑树存储空间是数组+链表的2倍（例如，树存储父、子节点，2个指针而不是1个）</li><li>泊松分布：链表长度为0的概率是60%，为1的概率是30%，为2的概率是7%…为8的概率几乎可以忽略，是非常极端的情况；并且达到8的时候，链表查询的平均开销<strong>数学上</strong>大于红黑树。</li></ul><h3 id="更简洁的搅动算法"><a class="markdownIt-Anchor" href="#更简洁的搅动算法"></a> 更简洁的搅动算法</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> <span class="type">int</span> <span class="title function_">hash</span><span class="params">(Object key)</span> &#123;</span><br><span class="line">    <span class="type">int</span> h;</span><br><span class="line">    <span class="keyword">return</span> (key == <span class="literal">null</span>) ? <span class="number">0</span> : (h = key.hashCode()) ^ (h &gt;&gt;&gt; <span class="number">16</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="懒初始化"><a class="markdownIt-Anchor" href="#懒初始化"></a> 懒初始化</h3><blockquote><p>jdk1.8只有在第一次调用<code>put()</code>时才真正为HashMap分配内存。</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> V <span class="title function_">putVal</span><span class="params">(<span class="type">int</span> hash, K key, V value, <span class="type">boolean</span> onlyIfAbsent, <span class="type">boolean</span> evict)</span> &#123;</span><br><span class="line">    Node&lt;K, V&gt;[] tab; Node&lt;K, V&gt; p; <span class="type">int</span> n, i;</span><br><span class="line">    <span class="comment">// 如果为空就初始化</span></span><br><span class="line">    <span class="keyword">if</span> ((tab = table) == <span class="literal">null</span> || (n = tab.length) == <span class="number">0</span>)</span><br><span class="line">        n = (tab = resize()).length;</span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="尾插法"><a class="markdownIt-Anchor" href="#尾插法"></a> 尾插法</h3><p>jdk1.8使用尾插法，避免头插法在并发下的死循环问题</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> V <span class="title function_">putVal</span><span class="params">(<span class="type">int</span> hash, K key, V value, <span class="type">boolean</span> onlyIfAbsent, <span class="type">boolean</span> evict)</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 使用for循环，binCount充当计数器，哈希冲突导致链表长度为8，就转为红黑树</span></span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">binCount</span> <span class="operator">=</span> <span class="number">0</span>; ; ++binCount) &#123;</span><br><span class="line">        <span class="comment">// 遍历到尾节点</span></span><br><span class="line">        <span class="keyword">if</span> ((e = p.next) == <span class="literal">null</span>) &#123;</span><br><span class="line">            p.next = newNode(hash, key, value, <span class="literal">null</span>);</span><br><span class="line">            <span class="keyword">if</span> (binCount &gt;= TREEIFY_THRESHOLD - <span class="number">1</span>)</span><br><span class="line">                <span class="comment">// 实际上会检查数组容量是否大于64，小于则只调用resize()</span></span><br><span class="line">                treeifyBin(tab, hash); </span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// 遍历过程中发现相同Key，那么跳出循环，在代码最后做一个替换操作，并返回oldValue</span></span><br><span class="line">        <span class="keyword">if</span> (e.hash == hash &amp;&amp; ((k = e.kay) == key || (key != <span class="literal">null</span> &amp;&amp; key.equals(k))))</span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        <span class="comment">// 否则 p = p.next</span></span><br><span class="line">        p = e;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="comment">// 替换返回旧值</span></span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">final</span> <span class="keyword">void</span> <span class="title function_">treeifyBin</span><span class="params">(Node&lt;K, V&gt;[] tab, <span class="type">int</span> hash)</span> &#123;</span><br><span class="line">    <span class="type">int</span> n, index; Node&lt;K, V&gt; e;</span><br><span class="line">    <span class="comment">// 数组长度小于64，只扩容不转红黑树</span></span><br><span class="line">    <span class="keyword">if</span> (tab == <span class="literal">null</span> || (n = tab.length) &lt; MIN_TREEIFY_CAPACITY)</span><br><span class="line">        resize();</span><br><span class="line">    <span class="keyword">else</span> ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><blockquote><p>注意，尾插法仍然不能保证原子性，在并发情况下仍然会发生更新丢失。</p></blockquote><h3 id="resize批量迁移"><a class="markdownIt-Anchor" href="#resize批量迁移"></a> resize批量迁移</h3><p>在链表情况下，hash后的槽位依然符合<a href="####rehash%E8%A7%84%E5%BE%8B">规律</a>：<br />rehash的结果要么和原来相同；要么是原来的位置+高位</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> Node&lt;K, V&gt;[] resize() &#123;</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 槽位不为空才需要迁移</span></span><br><span class="line">    <span class="keyword">if</span> ((e = oldTab[j]) != <span class="literal">null</span>) &#123;</span><br><span class="line">        oldTab[j] = <span class="literal">null</span>;</span><br><span class="line">        <span class="comment">// e.next为空，只有1个元素，直接迁移</span></span><br><span class="line">        <span class="keyword">if</span> (e.next == <span class="literal">null</span>)</span><br><span class="line">            <span class="comment">// hash并取模newCap，得到新槽位</span></span><br><span class="line">            newTab[e.hash &amp; (newCap - <span class="number">1</span>)] = e;</span><br><span class="line">        <span class="keyword">else</span> <span class="keyword">if</span> (e <span class="keyword">instanceof</span> TreeNode)</span><br><span class="line">            ((TreeNode&lt;K, V&gt;) e).split(<span class="built_in">this</span>, newTab, j, oldCap);</span><br><span class="line">        <span class="comment">// 槽位引用为列表</span></span><br><span class="line">        <span class="keyword">else</span> &#123;</span><br><span class="line">            <span class="comment">// 因为容量变了，hash()取模后不一定得到原来的槽位，需要全部重新hash</span></span><br><span class="line">            <span class="comment">// 要么hash和原来一致</span></span><br><span class="line">            Node&lt;K, V&gt; loHead = <span class="literal">null</span>, loTail = <span class="literal">null</span>;</span><br><span class="line">            <span class="comment">// 要么hash为原值 + 高位</span></span><br><span class="line">            Node&lt;K, V&gt; hiHead = <span class="literal">null</span>, hiTail = <span class="literal">null</span>;</span><br><span class="line">            Node&lt;K, V&gt; next;</span><br><span class="line"></span><br><span class="line">            <span class="keyword">do</span> &#123;</span><br><span class="line">                next = e.next;</span><br><span class="line">                <span class="comment">// hash后高位不为1，和原来一致</span></span><br><span class="line">                <span class="keyword">if</span> ((e.hash &amp; oldCap) == <span class="number">0</span>) &#123;</span><br><span class="line">                    <span class="keyword">if</span> (loTail == <span class="literal">null</span>)</span><br><span class="line">                        <span class="comment">// 初始化更新头节点</span></span><br><span class="line">                        loHead = e;</span><br><span class="line">                    <span class="keyword">else</span></span><br><span class="line">                        <span class="comment">// 其他时候更新尾节点</span></span><br><span class="line">                        loTail.next = e;</span><br><span class="line">                    <span class="comment">// loTail = loTail.next</span></span><br><span class="line">                    loTail = e;</span><br><span class="line">                &#125;</span><br><span class="line">                <span class="keyword">else</span> &#123;</span><br><span class="line">                    <span class="comment">// 同样的逻辑处理高位</span></span><br><span class="line">                &#125;</span><br><span class="line">            &#125; <span class="keyword">while</span> ((e = next) != <span class="literal">null</span>);</span><br><span class="line">            <span class="keyword">if</span> (loTail != <span class="literal">null</span>) &#123;</span><br><span class="line">                loTail.next = <span class="literal">null</span>;</span><br><span class="line">                newTab[j] = loHead;</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">if</span> (hiTail != <span class="literal">null</span>) &#123;</span><br><span class="line">                hiTail.next = <span class="literal">null</span>;</span><br><span class="line">                <span class="comment">// 容量都是2的幂次，等于+高位</span></span><br><span class="line">                newTab[j + oldCap] = hiHead;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    ...</span><br><span class="line">    <span class="keyword">return</span> newTab;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>对于树的迁移，因为hash取模newCapacity导致槽位变化，也要分割。分割逻辑和链表一样，只不过多了判断：</p><ol><li>分割后，是否需要退化为链表？红黑树节点少于6就退化；否则重新生成红黑树</li><li>如果全部hash取模后都在低位/高位，那直接迁移整棵红黑树。</li></ol><h1 id="concurrent-hashmap"><a class="markdownIt-Anchor" href="#concurrent-hashmap"></a> Concurrent HashMap</h1><blockquote><p>ConcurrentHashMap通过分段Segment锁控制并发级别，只对核心<code>put()</code>逻辑上锁，并且在获取锁失败时预创建节点，实现了高效的原子更新。</p></blockquote><h2 id="hashtable的性能问题"><a class="markdownIt-Anchor" href="#hashtable的性能问题"></a> HashTable的性能问题</h2><p>HashTable通过使用<code>synchronized</code>关键字来使<code>put()</code>同步。<br />这时使用<code>table.put(key, value)</code>，会直接为实例<code>table</code>上锁，从而保证<code>put()</code>原子性，解决并发下数据丢失问题。</p><p>然而，这样做<strong>锁的粒度太大</strong>了，</p><ol><li>一些不需要原子执行的操作（如<code>hash()</code>、<code>indexFor()</code>、潜在的<code>resize()</code>）都串行执行，</li><li>并且即使两个<code>put()</code>操作<strong>没有竞争也会被上锁</strong>，<br />因此，使用HashTable效率低下。</li></ol><h2 id="segment分段锁"><a class="markdownIt-Anchor" href="#segment分段锁"></a> Segment分段锁</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">Segment</span>&lt;K, V&gt; <span class="keyword">extends</span> <span class="title class_">ReentrantLock</span> <span class="keyword">implements</span> <span class="title class_">Serializable</span> &#123;</span><br><span class="line">    <span class="comment">// 注意，每个Segment代表HashEntry数组，因此叫做Segment段</span></span><br><span class="line">    <span class="keyword">transient</span> <span class="keyword">volatile</span> HashEntry&lt;K, V&gt;[] table;</span><br><span class="line">    <span class="keyword">transient</span> <span class="type">int</span> count;</span><br><span class="line">    <span class="keyword">transient</span> <span class="type">int</span> modCount;</span><br><span class="line">    <span class="keyword">transient</span> <span class="type">int</span> threshold;</span><br><span class="line">    <span class="keyword">final</span> <span class="type">float</span> loadFactor;</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">class</span> <span class="title class_">HashEntry</span>&lt;K, V&gt; &#123;</span><br><span class="line">    <span class="keyword">final</span> <span class="type">int</span> hash;</span><br><span class="line">    <span class="keyword">final</span> K key;</span><br><span class="line">    <span class="keyword">volatile</span> V value;</span><br><span class="line">    <span class="keyword">volatile</span> HashEntry&lt;K, V&gt; next;</span><br><span class="line"></span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>使用ConcurrentHashMap时，会这样执行<code>put()</code>：</p><ol><li>首先通过<code>hash()</code>得到槽位<code>index</code></li><li>对槽位（段）上锁，<code>segment[index].lock();</code></li><li>生成Entry，<code>new Entry(key, value, hashCode);</code></li><li>将Entry放入<code>Segment[]</code></li><li>释放槽位锁，<code>segment[index].unlock();</code></li></ol><p>通过这样的操作：</p><ol><li>保证了<code>putVal()</code>的原子性，解决了数据丢失问题</li><li>在两个线程hash槽位不同时，保证了并发性</li></ol><h2 id="可重入锁put"><a class="markdownIt-Anchor" href="#可重入锁put"></a> 可重入锁put</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">final</span> V <span class="title function_">put</span><span class="params">(K key, <span class="type">int</span> hash, V value, <span class="type">boolean</span> onlyIfAbsent)</span> &#123;</span><br><span class="line">    <span class="comment">// 上锁</span></span><br><span class="line">    HashEntry&lt;K, V&gt; node = tryLock() ? <span class="literal">null</span> :</span><br><span class="line">        <span class="comment">// 没有拿到锁的时候，如果没有相同Key，预先创建node</span></span><br><span class="line">        scanAndLockForPut(key, hash, value);</span><br><span class="line">    V oldValue;</span><br><span class="line">    <span class="keyword">try</span> &#123;</span><br><span class="line">        <span class="comment">// 遍历 HashEntry e</span></span><br><span class="line">            <span class="comment">// 发现相同Key，覆盖Value，并保存oldValue</span></span><br><span class="line">            <span class="comment">// 遍历完，e == null，插入新HashEntry</span></span><br><span class="line">            <span class="keyword">else</span> &#123;</span><br><span class="line">                <span class="comment">// 等待时已经创建好，直接插入node</span></span><br><span class="line">                <span class="keyword">if</span> (node != <span class="literal">null</span>) node.setNext(first);</span><br><span class="line">                <span class="comment">// jdk1.7 头插法</span></span><br><span class="line">                <span class="keyword">else</span> node = <span class="keyword">new</span> <span class="title class_">HashEntry</span>&lt;K, V&gt;(hash, key, value, first);</span><br><span class="line">                <span class="comment">// 其他逻辑，如扩容</span></span><br><span class="line">                ...</span><br><span class="line">            &#125;</span><br><span class="line">    &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">        unlock();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> oldValue;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="预创建node"><a class="markdownIt-Anchor" href="#预创建node"></a> 预创建node</h3><p>在<code>tryLock()</code>失败时，线程会尝试预创建node</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> HashEntry&lt;K, V&gt; <span class="title function_">scanAndLockForPut</span><span class="params">(K key, <span class="type">int</span> hash, V value)</span> &#123;</span><br><span class="line">    HashEntry&lt;K, V&gt; first = entryForHash(<span class="built_in">this</span>, hash);</span><br><span class="line">    HashEntry&lt;K, V&gt; e = first;</span><br><span class="line">    HashEntry&lt;K, V&gt; node = <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line">    <span class="type">int</span> <span class="variable">retries</span> <span class="operator">=</span> -<span class="number">1</span>;</span><br><span class="line">    <span class="comment">// 遍历HashEntry链表</span></span><br><span class="line">    <span class="keyword">while</span> (!tryLock()) &#123;</span><br><span class="line">        HashEntry&lt;K, V&gt; f;</span><br><span class="line">        <span class="keyword">if</span> (retries &lt; <span class="number">0</span>) &#123;</span><br><span class="line">            <span class="comment">// 遍历完Entry，尝试创建新节点</span></span><br><span class="line">            <span class="keyword">if</span> (e == <span class="literal">null</span>) &#123;</span><br><span class="line">                <span class="keyword">if</span> (node == <span class="literal">null</span>) &#123;</span><br><span class="line">                    <span class="comment">// 创建新节点</span></span><br><span class="line">                    node = <span class="keyword">new</span> <span class="title class_">HashEntry</span>&lt;K, V&gt;(hash, key, value, <span class="literal">null</span>);</span><br><span class="line">                &#125;</span><br><span class="line">                retries = <span class="number">0</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="comment">// 如果发现相同key，不创建</span></span><br><span class="line">            <span class="keyword">else</span> <span class="keyword">if</span> (key.equals(e.key)) retries = <span class="number">0</span>;</span><br><span class="line">            <span class="comment">// 遍历</span></span><br><span class="line">            <span class="keyword">else</span> e = e.next;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// 头节点有变化（新插入了节点），重置状态</span></span><br><span class="line">        <span class="keyword">else</span> <span class="keyword">if</span> ((retries &amp; <span class="number">1</span>) == <span class="number">0</span> &amp;&amp;</span><br><span class="line">                 (f = entryForHash(<span class="built_in">this</span>, hash)) != first) &#123;</span><br><span class="line">            e = first = f;</span><br><span class="line">            retries = -<span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// 饥饿，直接上锁</span></span><br><span class="line">        <span class="keyword">else</span> <span class="keyword">if</span> (++retries &gt; MAX_SCAN_RETRIES) &#123;</span><br><span class="line">            lock();</span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> node;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="concurrencylevel并发级别"><a class="markdownIt-Anchor" href="#concurrencylevel并发级别"></a> concurrencyLevel并发级别</h2><p>每个Segment有多少个Entry？ConcurrentHashMap通过<code>initialCapacity / concurrencyLevel</code>来控制。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">ConcurrentHashMap</span><span class="params">(<span class="type">int</span> initialCapacity, <span class="type">float</span> loadFactor, <span class="type">int</span> concurrencyLevel)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">ssize</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line">    <span class="comment">// 这里主要用于计算sshift，ssize最后是concurrencyLevel向上取2幂次</span></span><br><span class="line">    <span class="keyword">while</span> (ssize &lt; concurrencyLevel) &#123;</span><br><span class="line">        ++sshift;</span><br><span class="line">        ssize &lt;&lt;= <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    ...</span><br><span class="line">    <span class="comment">// 实际总容量initialCapacity = concurrencyLevel向上取2次幂 * 每段Entry数量</span></span><br><span class="line">    <span class="type">int</span> <span class="variable">c</span> <span class="operator">=</span> initialCapacity / ssize;</span><br><span class="line">    <span class="keyword">if</span> (c * ssize &lt; initalCapacity) ++c;</span><br><span class="line">    <span class="type">int</span> <span class="variable">cap</span> <span class="operator">=</span> MIN_SEGMENT_TABLE_CAPACITY;</span><br><span class="line">    <span class="comment">// cap = c向上取整2次幂</span></span><br><span class="line">    <span class="keyword">while</span> (cap &lt; c) cap &lt;&lt;= <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line">    Segment&lt;K, V&gt; s0 = </span><br><span class="line">        <span class="keyword">new</span> <span class="title class_">Segment</span>&lt;K, V&gt;(loadFactor, (<span class="type">int</span>)(cap * loadFactor), </span><br><span class="line">                          <span class="comment">// Segment容量为cap</span></span><br><span class="line">                          (HashEntry&lt;K, V&gt;[]) <span class="keyword">new</span> <span class="title class_">HashEntry</span>[cap]);</span><br><span class="line">    <span class="comment">// 2幂次Capacity</span></span><br><span class="line">    Segment&lt;K, V&gt;[] ss = (Segment&lt;K, V&gt;[]) <span class="keyword">new</span> <span class="title class_">Segment</span>[ssize];</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;代理&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#代理&quot;&gt;&lt;/a&gt; 代理&lt;/h1&gt;
&lt;blockquote&gt;
&lt;p&gt;代理对象通过&lt;code&gt;invoke&lt;/code&gt;，实现类与非核心功能的解耦。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;static&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;title function_&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(String[] args)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;type&quot;&gt;Payment&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;payment&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Payment&lt;/span&gt;(&lt;span class=&quot;string&quot;&gt;&amp;quot;AliPay&amp;quot;&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;type&quot;&gt;Pay&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;proxy&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; ProxyUtil.createProxy(payment);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    proxy.pay(amount);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Payment&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;implements&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Pay&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;meta&quot;&gt;@Overide&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;public&lt;/span&gt; payResp &lt;span class=&quot;title function_&quot;&gt;pay&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(BigDecimal payAmount)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;comment&quot;&gt;// Payment Business...&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;interface&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Pay&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;abstract&lt;/span&gt; payResp &lt;span class=&quot;title function_&quot;&gt;pay&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(BigDecimal payAmount)&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;ProxyUtils&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;static&lt;/span&gt; Pay &lt;span class=&quot;title function_&quot;&gt;createProxy&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(Payment payment)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; (Pay) Proxy.newProxyInstance(&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            ProxyUtil.class.getClassLoader(),&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Class&lt;/span&gt;[]&amp;#123; Pay.class &amp;#125;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;InvocationHandler&lt;/span&gt;() &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;meta&quot;&gt;@Overide&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;keyword&quot;&gt;public&lt;/span&gt; Object &lt;span class=&quot;title function_&quot;&gt;invoke&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(Object proxy, Method method, Object[] args)&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;throws&lt;/span&gt; Throwable &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    &lt;span class=&quot;comment&quot;&gt;// Payment Environment Init...&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    &lt;span class=&quot;comment&quot;&gt;// Payment Safe Guard...&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; method.invoke(payment, args);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        );&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;h1 id=&quot;反射&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#反射&quot;&gt;&lt;/a&gt; 反射&lt;/h1&gt;
&lt;blockquote&gt;
&lt;p&gt;反射允许对成员变量、成员方法、构造方法信息进行编程访问。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;例如，IDE的智能补全、参数提示，就是使用反射实现。&lt;/p&gt;
&lt;h2 id=&quot;class字节码获取&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#class字节码获取&quot;&gt;&lt;/a&gt; Class字节码获取&lt;/h2&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;Class clazz;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;clazz = Class.forName(&lt;span class=&quot;string&quot;&gt;&amp;quot;com.yourpackage.TargetClass&amp;quot;&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;// 参数&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;clazz = TargetClass.class;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;comment&quot;&gt;// 有实例时&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;clazz = instance.getClass();&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;h2 id=&quot;场景&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#场景&quot;&gt;&lt;/a&gt; 场景&lt;/h2&gt;
&lt;p&gt;反射可以与配置文件结合，从而动态地创建对象。例如&lt;code&gt;application.yml&lt;/code&gt;里数据库的配置、端口号等。&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;Properties&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;prop&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Properties&lt;/span&gt;();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;FileInputStream&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;fis&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;FileInputStream&lt;/span&gt;(ROOT + &lt;span class=&quot;string&quot;&gt;&amp;quot;src/main/resources/application.properties&amp;quot;&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;String&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;dataSourceUrl&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; (String) perp.get(&lt;span class=&quot;string&quot;&gt;&amp;quot;dataSource&amp;quot;&lt;/span&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;String&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;dbName&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; (String) extractDbName(dataSourceUrl);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;Class&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;clazz&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; Class.forName(dbName);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;Constructor&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;con&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; clazz.getDeclaredConstructor();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;Object&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;o&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; con.newInstance();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;...&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;h1 id=&quot;hashmap&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#hashmap&quot;&gt;&lt;/a&gt; HashMap&lt;/h1&gt;
&lt;h2 id=&quot;jdk17&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#jdk17&quot;&gt;&lt;/a&gt; jdk1.7&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;jdk1.7的HashMap数据结构是：数组 + 单向链表&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;当哈希后，得到的数组槽位已经存放了其他元素，这时候就需要运用指针在同一个槽位存放多个元素。&lt;/p&gt;
&lt;h2 id=&quot;头插法&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#头插法&quot;&gt;&lt;/a&gt; 头插法&lt;/h2&gt;
&lt;p&gt;jdk1.7使用的方法是头插法，这样就不需要遍历到链表尾部再插入，性能高。&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;title function_&quot;&gt;createEntry&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; hash, K key, V value, &lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; bucketIdx)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    Entry&amp;lt;K, V&amp;gt; e = table[bucketIdx];&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;comment&quot;&gt;// 这里使用头插法，在槽位头部插入新元素，并指向e，成为新的槽位引用&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    table[bucketIdx] = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;Entry&lt;/span&gt;&amp;lt;&amp;gt;(hash, key, value, e); &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    size++;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;title function_&quot;&gt;Entry&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; h, K k, V v, Entry&amp;lt;K, V&amp;gt; n)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;built_in&quot;&gt;this&lt;/span&gt;.value = v;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;built_in&quot;&gt;this&lt;/span&gt;.next = n;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;built_in&quot;&gt;this&lt;/span&gt;.key = k;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;built_in&quot;&gt;this&lt;/span&gt;.hash = h;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;这种方法需要最终更新槽位指向新插入的节点，否则单向链表找不到新插入的元素。&lt;/p&gt;
&lt;h2 id=&quot;利用2次方机器特性&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#利用2次方机器特性&quot;&gt;&lt;/a&gt; 利用2次方机器特性&lt;/h2&gt;</summary>
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>pandoc使用</title>
    <link href="http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/pandoc%E4%BD%BF%E7%94%A8/"/>
    <id>http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/pandoc%E4%BD%BF%E7%94%A8/</id>
    <published>2025-04-22T16:00:00.000Z</published>
    <updated>2025-05-27T02:47:21.445Z</updated>
    
    <content type="html"><![CDATA[<h1 id="文本格式转换"><a class="markdownIt-Anchor" href="#文本格式转换"></a> 文本格式转换</h1><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pandoc --from markdown --to docx source.md -o dest.docx</span><br><span class="line">pandoc -f markdown source.md -t docx -o dest.docx</span><br><span class="line">pandoc source.md -o dest.docx --ignore-args # 忽略参数</span><br></pre></td></tr></table></figure><blockquote><p><strong>注意</strong>：为了最佳转换效果，markdown文件每行后都要空行</p></blockquote><h1 id="模板"><a class="markdownIt-Anchor" href="#模板"></a> 模板</h1><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">pandoc --reference-doc template.docx source.md -o dest.docx</span><br></pre></td></tr></table></figure><h1 id="md2epub"><a class="markdownIt-Anchor" href="#md2epub"></a> md2epub</h1><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">首先把所有的md文件列出来</span></span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># 递归查找所有 .md 文件（排除 README.md 和 SUMMARY.md）</span></span></span><br><span class="line">find . -name &quot;*.md&quot; ! -name &quot;README.md&quot; ! -name &quot;SUMMARY.md&quot; | sort &gt; filelist.txt</span><br><span class="line"><span class="meta prompt_">#</span><span class="language-bash"><span class="comment"># 然后编辑 `filelist.txt`，确保文件顺序正确（例如按 `SUMMARY.md` 的目录结构排序）。</span></span></span><br><span class="line"></span><br><span class="line">pandoc --standalone --toc \</span><br><span class="line">--metadata title=&quot;MIT6.824 分布式系统&quot; \</span><br><span class="line">--metadata author=&quot;Robert Morris&quot; \</span><br><span class="line">-o output.epub $(cat filelist.txt) </span><br></pre></td></tr></table></figure><blockquote><p><strong>注意</strong>：对于gitbook，pandoc可能不能正确处理路径，推荐使用honkit。</p></blockquote><h2 id="honkit"><a class="markdownIt-Anchor" href="#honkit"></a> honkit</h2><figure class="highlight json"><table><tr><td class="code"><pre><span class="line"><span class="comment">// book.json</span></span><br><span class="line"><span class="punctuation">&#123;</span></span><br><span class="line">  <span class="attr">&quot;title&quot;</span><span class="punctuation">:</span> <span class="string">&quot;MIT6.824 分布式系统&quot;</span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;author&quot;</span><span class="punctuation">:</span> <span class="string">&quot;Robert Morris&quot;</span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;plugins&quot;</span><span class="punctuation">:</span> <span class="punctuation">[</span><span class="string">&quot;hints&quot;</span><span class="punctuation">]</span><span class="punctuation">,</span></span><br><span class="line">  <span class="attr">&quot;pluginsConfig&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">    <span class="attr">&quot;hints&quot;</span><span class="punctuation">:</span> <span class="punctuation">&#123;</span></span><br><span class="line">      <span class="attr">&quot;info&quot;</span><span class="punctuation">:</span> <span class="string">&quot;fa fa-info-circle&quot;</span><span class="punctuation">,</span></span><br><span class="line">      <span class="attr">&quot;warning&quot;</span><span class="punctuation">:</span> <span class="string">&quot;fa fa-exclamation-triangle&quot;</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line">  <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">&#125;</span></span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">安装honkit</span></span><br><span class="line">npm install honkit --save-dev</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">需要calibre转换</span></span><br><span class="line">ebook-convert --version</span><br><span class="line"></span><br><span class="line">npm init -y</span><br><span class="line">npx honkit epub ./ ./mybook.epub</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;文本格式转换&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#文本格式转换&quot;&gt;&lt;/a&gt; 文本格式转换&lt;/h1&gt;
&lt;figure class=&quot;highlight shell&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pr</summary>
      
    
    
    
    
    <category term="基本操作" scheme="http://simuleite.github.io/tags/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/"/>
    
  </entry>
  
  <entry>
    <title>ssh使用</title>
    <link href="http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/ssh%E4%BD%BF%E7%94%A8/"/>
    <id>http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/ssh%E4%BD%BF%E7%94%A8/</id>
    <published>2025-04-13T16:00:00.000Z</published>
    <updated>2025-04-25T06:34:47.753Z</updated>
    
    <content type="html"><![CDATA[<h1 id="ssh登录"><a class="markdownIt-Anchor" href="#ssh登录"></a> ssh登录</h1><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ssh &lt;user_name&gt;@&lt;remote_ip&gt; -p &lt;remote_port&gt; -i &lt;your_key&gt;</span><br></pre></td></tr></table></figure><h1 id="ssh端口映射"><a class="markdownIt-Anchor" href="#ssh端口映射"></a> ssh端口映射</h1><p>可以用于不保留端口的情况下，远程连接数据库等。</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">ssh -N -L &lt;local_port&gt;:localhost:&lt;remote_port&gt; &lt;user_name&gt;@&lt;remote_ip&gt; -p &lt;remote_port&gt; -i &lt;your_key&gt; </span><br></pre></td></tr></table></figure><h2 id="脚本批量映射"><a class="markdownIt-Anchor" href="#脚本批量映射"></a> 脚本批量映射</h2><p>需要注意，Nacos有gRPC，除了8848端口外，9848端口也要一起开放。</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">Port Mapping</span></span><br><span class="line">PORTS=(</span><br><span class="line">    &quot;ulocalport:localhost:uremoteport&quot;</span><br><span class="line">    # MySQL</span><br><span class="line">    &quot;53306:localhost:3306&quot;</span><br><span class="line">    # Nacos</span><br><span class="line">    &quot;58848:localhost:8848&quot;</span><br><span class="line">    &quot;59848:localhost:9848&quot;</span><br><span class="line">    # Redis</span><br><span class="line">    &quot;56379:localhost:6379&quot;</span><br><span class="line">    # RocketMQ namesrv</span><br><span class="line">    &quot;59876:localhost:9876&quot;</span><br><span class="line">    # RocketMQ broker</span><br><span class="line">    &quot;510911:localhost:10911&quot;</span><br><span class="line">)</span><br><span class="line"></span><br><span class="line">ARGS=()</span><br><span class="line">for port in &quot;$&#123;PORTS[@]&#125;&quot;; do</span><br><span class="line">    ARGS+=(-L &quot;$port&quot;)</span><br><span class="line">done</span><br><span class="line"></span><br><span class="line">ssh -o ServerAliveInterval=60 -N &quot;$&#123;ARGS[@]&#125;&quot; &lt;username&gt;@&lt;remote_ip&gt;</span><br></pre></td></tr></table></figure><h1 id="密钥登录"><a class="markdownIt-Anchor" href="#密钥登录"></a> 密钥登录</h1><ol><li>首先在本地生成一份密钥，然后将公钥上传到remote的<code>~/.ssh/authorized_keys</code></li><li>修改remote<code>/etc/ssh/sshd_config</code></li></ol><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">新端口</span></span><br><span class="line">Port 22</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">启用密钥认证</span></span><br><span class="line">PubkeyAuthentication yes</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">禁用密码登录</span></span><br><span class="line">PasswordAuthentication no</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">允许Root登录但禁止密码验证</span></span><br><span class="line">PermitRootLogin prohibit-password</span><br></pre></td></tr></table></figure><ol start="3"><li>重启ssh</li></ol><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">Ubuntu/Debian</span></span><br><span class="line">sudo systemctl restart ssh</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">CentOS/RHEL</span></span><br><span class="line">sudo systemctl restart sshd</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;ssh登录&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#ssh登录&quot;&gt;&lt;/a&gt; ssh登录&lt;/h1&gt;
&lt;figure class=&quot;highlight shell&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;</summary>
      
    
    
    
    
    <category term="基本操作" scheme="http://simuleite.github.io/tags/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/"/>
    
  </entry>
  
  <entry>
    <title>nginx使用</title>
    <link href="http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/nginx%E4%BD%BF%E7%94%A8/"/>
    <id>http://simuleite.github.io/ComputerScience/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/nginx%E4%BD%BF%E7%94%A8/</id>
    <published>2025-04-13T16:00:00.000Z</published>
    <updated>2025-04-28T07:09:14.843Z</updated>
    
    <content type="html"><![CDATA[<h1 id="新增配置"><a class="markdownIt-Anchor" href="#新增配置"></a> 新增配置</h1><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo vim /etc/nginx/sites-available/yourdomain.conf</span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">符号链接</span></span><br><span class="line">sudo ln -s /etc/nginx/sites-available/yourdomain.conf /etc/nginx/sites-enabled/</span><br></pre></td></tr></table></figure><h1 id="port2domain"><a class="markdownIt-Anchor" href="#port2domain"></a> port2domain</h1><h2 id="后端服务"><a class="markdownIt-Anchor" href="#后端服务"></a> 后端服务</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">server &#123;</span><br><span class="line">    listen 80;</span><br><span class="line">    server_name yourdomain.com www.yourdomain.com;</span><br><span class="line"></span><br><span class="line">    location / &#123;</span><br><span class="line">        proxy_pass http://127.0.0.1:9000;  # 项目运行的端口</span><br><span class="line">        proxy_set_header Host $host;</span><br><span class="line">        proxy_set_header X-Real-IP $remote_addr;</span><br><span class="line">        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;</span><br><span class="line">        proxy_set_header X-Forwarded-Proto $scheme;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    # 日志配置（可选）</span><br><span class="line">    error_log /var/log/nginx/yourdomain.error.log;</span><br><span class="line">    access_log /var/log/nginx/yourdomain.access.log;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="启用"><a class="markdownIt-Anchor" href="#启用"></a> 启用</h3><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">sudo nginx -t  # 检查配置是否正确</span><br><span class="line">sudo nginx -s reload</span><br></pre></td></tr></table></figure><h2 id="前端打包文件"><a class="markdownIt-Anchor" href="#前端打包文件"></a> 前端打包文件</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">location / &#123;</span><br><span class="line">    root /var/www/railcloud/dist;</span><br><span class="line">    index index.html;</span><br><span class="line">    try_files $uri $uri/ /index.html; # 支持SPA路由</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="启用-2"><a class="markdownIt-Anchor" href="#启用-2"></a> 启用</h3><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_"># </span><span class="language-bash">③ 设置正确权限</span></span><br><span class="line">sudo chown -R www-data:www-data /var/www/railcloud</span><br><span class="line">sudo chmod -R 755 /var/www/railcloud</span><br><span class="line"></span><br><span class="line">sudo nginx -t &amp;&amp; nginx -s reload</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;新增配置&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#新增配置&quot;&gt;&lt;/a&gt; 新增配置&lt;/h1&gt;
&lt;figure class=&quot;highlight shell&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;spa</summary>
      
    
    
    
    
    <category term="基本操作" scheme="http://simuleite.github.io/tags/%E5%9F%BA%E6%9C%AC%E6%93%8D%E4%BD%9C/"/>
    
  </entry>
  
  <entry>
    <title>ThreadPool线程池</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/ThreadPool%E7%BA%BF%E7%A8%8B%E6%B1%A0/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/ThreadPool%E7%BA%BF%E7%A8%8B%E6%B1%A0/</id>
    <published>2025-04-09T16:00:00.000Z</published>
    <updated>2025-04-24T08:58:31.862Z</updated>
    
    <content type="html"><![CDATA[<h1 id="线程资源通过线程池提供"><a class="markdownIt-Anchor" href="#线程资源通过线程池提供"></a> 线程资源通过线程池提供</h1><p>线程池可以减少创建、销毁线程的开销，解决资源不足问题。<br />如果手动创建线程，容易造成系统存在大量同类线程而导致内存耗尽、过度切换问题。</p><h1 id="不使用executors"><a class="markdownIt-Anchor" href="#不使用executors"></a> 不使用Executors</h1><p><code>Executors.newFixedThreadPool()</code> 固定大小线程池<br /><code>Executors.newSingleThreadExecutor()</code> 单线程池<br /><code>Executors.newCachedThreadPool()</code> 动态线程池</p><blockquote><p>Executors底层仍然使用<code>new</code>来创建线程，容易造成OOM。</p></blockquote><h2 id="自己调用threadpoolexecutor"><a class="markdownIt-Anchor" href="#自己调用threadpoolexecutor"></a> 自己调用ThreadPoolExecutor</h2><p>推荐的方法使，直接自己调用<code>new ThreadPoolExecutor</code>来创建线程池，设置自己的参数</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="title function_">ThreadPoolExecutor</span><span class="params">(</span></span><br><span class="line"><span class="params">    <span class="meta">@Range(from = 0, to = Integer.MAX_VALUE)</span> <span class="type">int</span>  corePoolSize,</span></span><br><span class="line"><span class="params">    <span class="meta">@Range(from = 1, to = Integer.MAX_VALUE)</span> <span class="type">int</span>  maximumPoolSize,</span></span><br><span class="line"><span class="params">    <span class="meta">@Range(from = 0, to = Long.MAX_VALUE)</span>    <span class="type">long</span> keepAliveTime,</span></span><br><span class="line"><span class="params">    <span class="meta">@NotNull</span> TimeUnit unit,</span></span><br><span class="line"><span class="params">    <span class="meta">@NotNull</span> BlockingQueue&lt;Runnable&gt; workQueue,</span></span><br><span class="line"><span class="params">    <span class="meta">@NotNull</span> ThreadFactory threadFactory,</span></span><br><span class="line"><span class="params">    <span class="meta">@NotNull</span> RejectedExecutionHandler handler</span></span><br><span class="line"><span class="params">)</span> &#123; ... &#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// Example</span></span><br><span class="line"><span class="type">ThreadPoolExecutor</span> <span class="variable">threadPool</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>(</span><br><span class="line">    <span class="number">5</span>,</span><br><span class="line">    <span class="number">10</span>,</span><br><span class="line">    <span class="number">0L</span>,</span><br><span class="line">    TimeUnit.SECONDS,</span><br><span class="line">    <span class="keyword">new</span> <span class="title class_">ArrayBlockingQueue</span>&lt;Runnable&gt;(<span class="number">10</span>), <span class="comment">// 10为容量</span></span><br><span class="line">    Executors.defaultThreadFactory(),</span><br><span class="line">    <span class="keyword">new</span> <span class="title class_">ThreadPoolExecutor</span>.AbortPolicy()</span><br><span class="line">);</span><br><span class="line"></span><br></pre></td></tr></table></figure><p><code>corePoolSize</code> 核心线程数量。<br /><code>maximumPoolSize</code> 最大线程数量（核心 + 临时）。只有核心线程满了，而且阻塞队列也满了，才会创建临时线程。<br /><code>keepAliveTime</code> 临时线程的空闲存活时间。<br /><code>threadFactory</code> 线程工厂，即以什么方式创建线程。<br /><code>handler</code> 核心线程、阻塞队列、临时线程都满了，触发拒绝策略。</p><p><code>Executors</code>的问题在于，它指定的阻塞队列大小是<code>Integer.MAX_VALUE</code>，会导致内存溢出；而<code>Executors.newCachedThreadPool()</code>更是指定了<code>maximumPoolSize</code>为<code>Integer.MAX_VALUE</code>。</p><h2 id="拒绝策略如何保证线程不丢"><a class="markdownIt-Anchor" href="#拒绝策略如何保证线程不丢"></a> 拒绝策略，如何保证线程不丢</h2><p>使用直接拒绝<code>AbortPolicy</code>策略，线程会丢失。此时可以使用</p><ul><li><code>CallerRunsPolicy</code> 直接在主线程同步执行（可能会阻塞主线程）</li><li><code>DiscardOldestPolicy</code> 将队列头部删除，新线程尾部入队（保证新任务优先级）</li><li><code>DiscardPolicy</code> 可以自己拓展，例如将任务放到redis、rocketmq中</li></ul><h2 id="threadpool工作流程"><a class="markdownIt-Anchor" href="#threadpool工作流程"></a> ThreadPool工作流程</h2><p>当前线程数没有达到<code>corePoolSize</code>之前，每个新任务都会触发创建新线程；<br />达到以后，才会放到阻塞队列里，等待线程任务执行完成，分发任务给核心线程。<br />阻塞队列满了，才会创建临时线程执行任务。</p><h1 id="threadpool如何初始化"><a class="markdownIt-Anchor" href="#threadpool如何初始化"></a> ThreadPool如何初始化？</h1><p>供参考的经验值：</p><ul><li>CPU密集任务（如数据统计、排序）：<br />核心线程 = 最大线程 = 核心数 + 1<br />减少上下文切换开销</li><li>IO密集任务：<br />核心线程 = 核心数 * 2；最大线程 = 核心数 * 4</li></ul><h1 id="动态线程池"><a class="markdownIt-Anchor" href="#动态线程池"></a> 动态线程池</h1>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;线程资源通过线程池提供&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#线程资源通过线程池提供&quot;&gt;&lt;/a&gt; 线程资源通过线程池提供&lt;/h1&gt;
&lt;p&gt;线程池可以减少创建、销毁线程的开销，解决资源不足问题。&lt;br /&gt;
如果手动创建线程，容易</summary>
      
    
    
    
    
    <category term="笔记" scheme="http://simuleite.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
    
  </entry>
  
  <entry>
    <title>JUC并发编程</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/JUC%E5%B9%B6%E5%8F%91%E7%BC%96%E7%A8%8B/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/JUC%E5%B9%B6%E5%8F%91%E7%BC%96%E7%A8%8B/</id>
    <published>2025-03-21T16:00:00.000Z</published>
    <updated>2025-03-25T12:12:16.682Z</updated>
    
    <content type="html"><![CDATA[<h1 id="java多线程"><a class="markdownIt-Anchor" href="#java多线程"></a> Java多线程</h1><p>回顾：操作系统的<a href="obsidian://open?vault=Obsidian%20Vault&amp;file=ComputerScience%2F%E7%9F%A5%E8%AF%86%2FOS%20%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F">进程</a>概念。进程的问题：上下文切换开销。为了解决这个问题，出现了线程。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Thread</span> <span class="variable">thread</span> <span class="operator">=</span> (() -&gt; &#123;</span><br><span class="line">    <span class="comment">// ...</span></span><br><span class="line">    System.out.pirntln(<span class="string">&quot;Sub Thread&quot;</span>);</span><br><span class="line">&#125;);</span><br><span class="line">thread.start();</span><br><span class="line">System.out.println(<span class="string">&quot;Main Thread&quot;</span>);</span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line">Main Thread # 主线程先输出结果，说明两个线程同时运行！</span><br><span class="line">Sub Thread</span><br></pre></td></tr></table></figure><h2 id="线程优先级"><a class="markdownIt-Anchor" href="#线程优先级"></a> 线程优先级</h2><p>Java使用抢占式调度，有以下三种优先级</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">MIN_PRIORITY</span><br><span class="line">MAX_PRIORITY</span><br><span class="line">NOM_PRIORITY</span><br></pre></td></tr></table></figure><h2 id="线程同步"><a class="markdownIt-Anchor" href="#线程同步"></a> 线程同步</h2><p>共享内存会出现缓存一致性问题，因此需要<strong>线程锁</strong>机制保证数据安全性（原子性）。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">synchronized</span> <span class="comment">// 悲观锁</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">synchronized</span> (Class.class / <span class="built_in">this</span>) &#123; ... &#125; <span class="comment">// 类锁</span></span><br><span class="line"><span class="keyword">synchronized</span> (<span class="keyword">new</span> <span class="title class_">Class</span>() / instanceOfClass) &#123; ... &#125; <span class="comment">// 实例锁</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">void</span> <span class="title function_">operation</span><span class="params">()</span> &#123; ... &#125;</span><br></pre></td></tr></table></figure><h1 id="锁"><a class="markdownIt-Anchor" href="#锁"></a> 锁</h1><p><code>synchronized</code>使用的锁存储在Java对象头中。</p><h3 id="重量级锁"><a class="markdownIt-Anchor" href="#重量级锁"></a> 重量级锁</h3><p>JDK6以前，<code>synchronized</code>被称为重量级锁。因为Java的线程是映射在OS原生线程上，上下文切换成本高；直到JDK6以后才优化了锁的实现。<br />简单来说，每个等待锁都会被封装成ObjectWaiter对象，分为三个区域</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">direction: right</span><br><span class="line">Entry Set -&gt; The Owner</span><br><span class="line">The Owner &lt;-&gt; Wait Set</span><br></pre></td></tr></table></figure><p>Entry Set会排队，直到它成为The Owner，享有资源。当The Owner调用<code>wait()</code>方法，就会挂起进入Wait Set，直到<code>wait()</code>所等待的操作完成。<br />但是每个线程占用同步代码块的时间并不长，完全不需要挂起又唤醒。<br />因此，可以使用<strong>自旋锁</strong></p><h3 id="自旋锁"><a class="markdownIt-Anchor" href="#自旋锁"></a> 自旋锁</h3><p>调用<code>wait()</code>，自旋锁并不是被挂起，而是无限循环是否能够获取锁；当等待时间太长，会恢复重量锁机制。<br />JDK6以后，自旋时间是动态变化的。如果某个线程经常自旋失败，它会直接使用重量级锁；反之，则会延长自旋时间。</p><h2 id="轻量级锁"><a class="markdownIt-Anchor" href="#轻量级锁"></a> 轻量级锁</h2><p>JDK6后，为了减少获得和释放锁的消耗，引入了轻量级锁。<br />轻量级锁的设计目标是，在无竞争状态下减少重量级锁带来的性能消耗（切换内核态、线程阻塞引发线程切换）。</p><blockquote><p>如果只有一个线程占用资源，那就不要加锁、解锁。<br />轻量级锁需要向系统申请互斥量。</p></blockquote><h3 id="cas算法"><a class="markdownIt-Anchor" href="#cas算法"></a> CAS算法</h3><p>Compare and Swap<br />CAS算法不是加锁，而是通过比较来判断对象是否已被修改，如果没有直接替换；如果被修改，那么修改失败。</p><p>轻量级锁就是使用CAS算法，如果CAS失败，那么进入重量级锁状态。</p><h2 id="偏向锁"><a class="markdownIt-Anchor" href="#偏向锁"></a> 偏向锁</h2><p>Biased Locking<br /><code>-XX:UserBiasLock</code><br />当只有一个线程反复访问同步代码块，JVM直接让该线程获取锁，避免不必要的不同步操作。<br />根据对象头底层数据结构，如果对象调用过<code>hashCode()</code>通过哈希值来检查一致性，那么对象头就没有空间存放ThreadId了（JVM通过这个id判断是否频繁访问），此时该线程只能使用轻量级锁。</p><h2 id="锁消除和锁粗化"><a class="markdownIt-Anchor" href="#锁消除和锁粗化"></a> 锁消除和锁粗化</h2><p>如果在运行过程中，根本没有出现资源竞争，那就会直接把锁消除掉。<br />如果某个资源频繁地开锁解锁（比如在循环内部<code>synchronized</code>），JVM会把锁的范围放大，避免加锁解锁的开销。</p><h1 id="java-memory-model"><a class="markdownIt-Anchor" href="#java-memory-model"></a> Java Memory Model</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Java Thread 1 &lt;-&gt; Working Memory 1 &lt;-&gt; Save/Load Operation </span><br><span class="line">Java Thread 2 &lt;-&gt; Working Memory 2 &lt;-&gt; Save/Load Operation </span><br><span class="line">Java Thread 3 &lt;-&gt; Working Memory 3 &lt;-&gt; Save/Load Operation </span><br><span class="line">Save/Load Operation &lt;-&gt; Main Memory</span><br></pre></td></tr></table></figure><p>JMM内存模型中有以下规定：</p><ol><li>所有变量存储在主内存</li><li>每条线程有自己的工作内存，不能直接操作主内存</li><li>不同线程间互相隔离，要传递内容，必须通过主内存</li></ol><h1 id="volatile"><a class="markdownIt-Anchor" href="#volatile"></a> volatile</h1><p><code>volatile</code>的最大作用是保证变量<strong>可见性</strong>，即发生修改后强制刷新到主内存中，使其他线程的缓存失效；相当于通知了其他线程要更新变量为最新版本。<br />注意，<code>volatile</code>不能保证原子性。</p><h1 id="lockcondition"><a class="markdownIt-Anchor" href="#lockcondition"></a> Lock&amp;Condition</h1><p>Lock用法：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Lock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLock</span>(); <span class="comment">// 可重入锁</span></span><br><span class="line"><span class="type">Runnable</span> <span class="variable">action</span> <span class="operator">=</span> () -&gt; &#123;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; <span class="number">100000</span>; i += <span class="number">1</span>) &#123;</span><br><span class="line">        lock.lock();</span><br><span class="line">        i += <span class="number">1</span>; <span class="comment">// 保证同一时刻只有一个线程操作i</span></span><br><span class="line">        lock.unlock();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">new</span> <span class="title class_">Thread</span>(action).start();</span><br><span class="line"><span class="keyword">new</span> <span class="title class_">Thread</span>(action).start();</span><br></pre></td></tr></table></figure><p>Condition用法：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Condition</span> <span class="variable">cond</span> <span class="operator">=</span> lock.newCondition();</span><br><span class="line"></span><br><span class="line"><span class="type">Thread</span> <span class="variable">thread1</span> <span class="operator">=</span> () -&gt; &#123;</span><br><span class="line">    ...</span><br><span class="line">    cond.await(); <span class="comment">// 等待</span></span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br><span class="line"><span class="type">Thread</span> <span class="variable">thread2</span> <span class="operator">=</span> () -&gt; &#123;</span><br><span class="line">    ...</span><br><span class="line">    cond.signal(); <span class="comment">// 唤醒await线程</span></span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">new</span> <span class="title class_">Thread</span>(thread1).start();</span><br><span class="line"><span class="keyword">new</span> <span class="title class_">Thread</span>(thread2).start();</span><br></pre></td></tr></table></figure><h2 id="leetcode-1114-顺序打印123"><a class="markdownIt-Anchor" href="#leetcode-1114-顺序打印123"></a> LeetCode 1114 顺序打印123</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Foo</span> &#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="type">Lock</span> <span class="variable">lock</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLock</span>();</span><br><span class="line">    <span class="keyword">private</span> <span class="type">Condition</span> <span class="variable">cond1</span> <span class="operator">=</span> lock.newCondition();</span><br><span class="line">    <span class="keyword">private</span> <span class="type">Condition</span> <span class="variable">cond2</span> <span class="operator">=</span> lock.newCondition();</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">volatile</span> <span class="type">int</span> <span class="variable">state</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="title function_">Foo</span><span class="params">()</span> &#123; &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">first</span><span class="params">(Runnable printFirst)</span> <span class="keyword">throws</span> InterruptedException &#123;</span><br><span class="line">        lock.lock();</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            <span class="comment">// printFirst.run() outputs &quot;first&quot;. Do not change or remove this line.</span></span><br><span class="line">            printFirst.run();</span><br><span class="line">            state = <span class="number">1</span>;</span><br><span class="line">            cond1.signal();</span><br><span class="line">        &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">            lock.unlock();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">second</span><span class="params">(Runnable printSecond)</span> <span class="keyword">throws</span> InterruptedException &#123;</span><br><span class="line">        lock.lock();</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            <span class="keyword">while</span> (state != <span class="number">1</span>) &#123;</span><br><span class="line">                cond1.await();</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="comment">// printSecond.run() outputs &quot;second&quot;. Do not change or remove this line.</span></span><br><span class="line">            printSecond.run();</span><br><span class="line">            state = <span class="number">2</span>;</span><br><span class="line">            cond2.signal();</span><br><span class="line">        &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">            lock.unlock();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">third</span><span class="params">(Runnable printThird)</span> <span class="keyword">throws</span> InterruptedException &#123;</span><br><span class="line">        lock.lock();</span><br><span class="line">        <span class="keyword">try</span> &#123;</span><br><span class="line">            <span class="keyword">while</span> (state != <span class="number">2</span>) &#123;</span><br><span class="line">                cond2.await();</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="comment">// printThird.run() outputs &quot;third&quot;. Do not change or remove this line.</span></span><br><span class="line">            printThird.run();</span><br><span class="line">        &#125; <span class="keyword">finally</span> &#123;</span><br><span class="line">            lock.unlock();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="可重入锁"><a class="markdownIt-Anchor" href="#可重入锁"></a> 可重入锁</h2><p>Re-entrant-Lock<br />这种锁可以多次加锁，同时也要多次解锁才算真的解锁了。</p><blockquote><p>可重入锁是一种排他锁，其他线程必须等锁释放了才可以获取到锁。</p></blockquote><h3 id="公平锁与非公平锁"><a class="markdownIt-Anchor" href="#公平锁与非公平锁"></a> 公平锁与非公平锁</h3><ul><li>公平锁：按照申请锁的时间去获得锁，会进入队列排队</li><li>非公平：抢占式获取锁</li></ul><h2 id="读写锁"><a class="markdownIt-Anchor" href="#读写锁"></a> 读写锁</h2><blockquote><p>读写锁在同一时刻，可以让多个线程获取到锁。</p></blockquote><ul><li>读锁：没有线程占用<strong>写锁</strong>的情况下，同一时间可以有多个线程加读锁。</li><li>写锁：没有线程占用<strong>读锁</strong>的情况下，只有一个线程可以加写锁。</li></ul><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">Lock</span> <span class="variable">reEntLock</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">ReentrantLockReadWriteLock</span>();</span><br><span class="line">reEntLock.readLock().lock();</span><br><span class="line">reEntLock.writeLock().lock();</span><br></pre></td></tr></table></figure><h2 id="锁降级-锁升级"><a class="markdownIt-Anchor" href="#锁降级-锁升级"></a> 锁降级、锁升级</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">reEntLock.writeLock().lock();</span><br><span class="line"><span class="comment">// 先加写锁，后加读锁，降级</span></span><br><span class="line">reEntLock.readLock().lock();</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line"></span><br><span class="line"><span class="comment">// 然后释放写锁，只留下读锁，锁降级</span></span><br><span class="line">reEntLock.writeLock().unlock();</span><br></pre></td></tr></table></figure><h2 id="aqs实现"><a class="markdownIt-Anchor" href="#aqs实现"></a> AQS实现</h2><p>Abstract Queued Synchronizer<br />在AQS中，一个线程获取锁后，其他线程进入等待队列。<br />等待队列由双向链表实现。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;java多线程&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#java多线程&quot;&gt;&lt;/a&gt; Java多线程&lt;/h1&gt;
&lt;p&gt;回顾：操作系统的&lt;a href=&quot;obsidian://open?vault=Obsidian%20Vault&amp;a</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>蓝桥杯 错题本</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/%E9%94%99%E9%A2%98%E6%9C%AC/%E8%93%9D%E6%A1%A5%E6%9D%AF%20%E9%94%99%E9%A2%98%E6%9C%AC/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/%E9%94%99%E9%A2%98%E6%9C%AC/%E8%93%9D%E6%A1%A5%E6%9D%AF%20%E9%94%99%E9%A2%98%E6%9C%AC/</id>
    <published>2025-03-18T16:00:00.000Z</published>
    <updated>2025-03-19T10:48:35.665Z</updated>
    
    <content type="html"><![CDATA[<h1 id="3513-岛屿个数"><a class="markdownIt-Anchor" href="#3513-岛屿个数"></a> 3513 岛屿个数</h1><p>#外岛数量 #bfs</p><h2 id="杰克船长算法"><a class="markdownIt-Anchor" href="#杰克船长算法"></a> 杰克船长算法</h2><blockquote><p>杰克船长在公海上游荡，每发现一处岛屿，他就会绕着岛走一圈，并把这个岛标记到地图上。</p></blockquote><p>这个问题的解决方法就在这里：我们<strong>一定</strong>要有一片完全连通的公海，只有在公海上遇到岛屿，才标记岛屿数量；绝不踏入内海。</p><p>可是测试用例是这样的：</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">5 5</span><br><span class="line">01111</span><br><span class="line">11001</span><br><span class="line">10101</span><br><span class="line">10001</span><br><span class="line">11111</span><br></pre></td></tr></table></figure><p>这个测试用例，只有<code>(0, 0)</code>是公海，怎么办呢？<br />我们用一圈公海把测试用例包围起来：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">processInput</span><span class="params">(Scanner sc)</span> &#123;</span><br><span class="line">    M = sc.nextInt();</span><br><span class="line">    N = sc.nextInt();</span><br><span class="line">    sc.nextLine();</span><br><span class="line">    map = <span class="keyword">new</span> <span class="title class_">int</span>[M + <span class="number">2</span>][N + <span class="number">2</span>]; <span class="comment">// 注意+2，多一圈&#x27;0&#x27;表示公海</span></span><br><span class="line">    visitedSea = <span class="keyword">new</span> <span class="title class_">boolean</span>[M + <span class="number">2</span>][N + <span class="number">2</span>];</span><br><span class="line">    visitedIsland = <span class="keyword">new</span> <span class="title class_">boolean</span>[M + <span class="number">2</span>][N + <span class="number">2</span>];</span><br><span class="line">    cnt = <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">x</span> <span class="operator">=</span> <span class="number">1</span>; x &lt;= M; x += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="type">String</span> <span class="variable">line</span> <span class="operator">=</span> sc.nextLine();</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">y</span> <span class="operator">=</span> <span class="number">1</span>; y &lt;= N; y += <span class="number">1</span>) &#123;</span><br><span class="line">            map[x][y] = line.charAt(y - <span class="number">1</span>) - <span class="string">&#x27;0&#x27;</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><span id="more"></span><p>接着就是对最外面一圈公海进行<code>bfs</code>遍历，只有在公海遇到岛屿才上岛：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">bfsSea</span><span class="params">(<span class="type">int</span> x, <span class="type">int</span> y)</span> &#123;</span><br><span class="line">    Queue&lt;Point&gt; queue = <span class="keyword">new</span> <span class="title class_">LinkedList</span>&lt;&gt;();</span><br><span class="line">    queue.add(<span class="keyword">new</span> <span class="title class_">Point</span>(x, y));</span><br><span class="line">    visitedSea[x][y] = <span class="literal">true</span>;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (!queue.isEmpty()) &#123;</span><br><span class="line">        <span class="type">Point</span> <span class="variable">point</span> <span class="operator">=</span> queue.poll();</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">d</span> <span class="operator">=</span> <span class="number">0</span>; d &lt; <span class="number">8</span>; d += <span class="number">1</span>) &#123; <span class="comment">// 8个方向！</span></span><br><span class="line">            <span class="type">int</span> <span class="variable">nx</span> <span class="operator">=</span> point.x + dx[d];</span><br><span class="line">            <span class="type">int</span> <span class="variable">ny</span> <span class="operator">=</span> point.y + dy[d];</span><br><span class="line">            <span class="keyword">if</span> (!isOutBound(nx, ny)) &#123;</span><br><span class="line">                <span class="keyword">if</span> (!visitedSea[nx][ny] &amp;&amp; map[nx][ny] == <span class="number">0</span>) &#123;</span><br><span class="line">                    visitedSea[nx][ny] = <span class="literal">true</span>;</span><br><span class="line">                    queue.add(<span class="keyword">new</span> <span class="title class_">Point</span>(nx, ny));</span><br><span class="line">                &#125; <span class="keyword">else</span> <span class="keyword">if</span> (!visitedIsland[nx][ny] &amp;&amp; map[nx][ny] == <span class="number">1</span>) &#123;</span><br><span class="line">                    cnt++;</span><br><span class="line">                    bfsIsland(nx, ny);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>注意测试用例的边界情况</p><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">5 6</span><br><span class="line">111111</span><br><span class="line">100001</span><br><span class="line">010101</span><br><span class="line">100001</span><br><span class="line">111111</span><br></pre></td></tr></table></figure><p>上面这个测试用例告诉我们：公海可以朝8个方向通行。而岛屿我们只朝4面通行</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">bfsIsland</span><span class="params">(<span class="type">int</span> x, <span class="type">int</span> y)</span> &#123;</span><br><span class="line">    Queue&lt;Point&gt; queue = <span class="keyword">new</span> <span class="title class_">LinkedList</span>&lt;&gt;();</span><br><span class="line">    queue.add(<span class="keyword">new</span> <span class="title class_">Point</span>(x, y));</span><br><span class="line">    visitedIsland[x][y] = <span class="literal">true</span>;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (!queue.isEmpty()) &#123;</span><br><span class="line">        <span class="type">Point</span> <span class="variable">point</span> <span class="operator">=</span> queue.poll();</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">d</span> <span class="operator">=</span> <span class="number">0</span>; d &lt; <span class="number">4</span>; d += <span class="number">1</span>) &#123; <span class="comment">// 4个方向</span></span><br><span class="line">            <span class="type">int</span> <span class="variable">nx</span> <span class="operator">=</span> point.x + dx[d];</span><br><span class="line">            <span class="type">int</span> <span class="variable">ny</span> <span class="operator">=</span> point.y + dy[d];</span><br><span class="line">            <span class="keyword">if</span> (!isOutBound(nx, ny) &amp;&amp; !visitedIsland[nx][ny] &amp;&amp; map[nx][ny] == <span class="number">1</span>) &#123;</span><br><span class="line">                visitedIsland[nx][ny] = <span class="literal">true</span>;</span><br><span class="line">                queue.add(<span class="keyword">new</span> <span class="title class_">Point</span>(nx, ny));</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>完整调用如下：<br />如果WA，可以把访问点全部打印出来，看看是不是代码有漏洞导致没遍历完</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="type">int</span>[][] map;</span><br><span class="line"><span class="keyword">static</span> <span class="type">boolean</span>[][] visitedSea;</span><br><span class="line"><span class="keyword">static</span> <span class="type">boolean</span>[][] visitedIsland;</span><br><span class="line"><span class="keyword">static</span> <span class="type">int</span> M, N;</span><br><span class="line"><span class="keyword">static</span> <span class="type">int</span> cnt;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">solution</span><span class="params">()</span> &#123;</span><br><span class="line">    List&lt;Integer&gt; ans = <span class="keyword">new</span> <span class="title class_">ArrayList</span>&lt;&gt;();</span><br><span class="line">    <span class="type">Scanner</span> <span class="variable">sc</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Scanner</span>(System.in);</span><br><span class="line">    <span class="type">int</span> <span class="variable">T</span> <span class="operator">=</span> sc.nextInt();</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; T; i += <span class="number">1</span>) &#123;</span><br><span class="line">        processInput(sc);</span><br><span class="line">        <span class="comment">// printMap();</span></span><br><span class="line"></span><br><span class="line">        <span class="comment">// 外面一圈都是公海，所以从(0, 0)开始就可以遍历整个公海</span></span><br><span class="line">        bfsSea(<span class="number">0</span>, <span class="number">0</span>); </span><br><span class="line">        ans.add(cnt);</span><br><span class="line">        <span class="comment">// printVisited();</span></span><br><span class="line">    &#125;</span><br><span class="line">    sc.close();</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> answer : ans) &#123;</span><br><span class="line">        System.out.println(answer);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">class</span> <span class="title class_">Point</span> &#123;</span><br><span class="line">    <span class="type">int</span> x;</span><br><span class="line">    <span class="type">int</span> y;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="title function_">Point</span><span class="params">(<span class="type">int</span> x, <span class="type">int</span> y)</span> &#123;</span><br><span class="line">        <span class="built_in">this</span>.x = x;</span><br><span class="line">        <span class="built_in">this</span>.y = y;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 上、下、左、右、左上、左下、右上、右下</span></span><br><span class="line"><span class="keyword">static</span> <span class="type">int</span>[] dx = &#123; -<span class="number">1</span>, <span class="number">1</span>, <span class="number">0</span>, <span class="number">0</span>, -<span class="number">1</span>, <span class="number">1</span>, -<span class="number">1</span>, <span class="number">1</span> &#125;;</span><br><span class="line"><span class="keyword">static</span> <span class="type">int</span>[] dy = &#123; <span class="number">0</span>, <span class="number">0</span>, -<span class="number">1</span>, <span class="number">1</span>, -<span class="number">1</span>, -<span class="number">1</span>, <span class="number">1</span>, <span class="number">1</span> &#125;;</span><br></pre></td></tr></table></figure>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;3513-岛屿个数&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#3513-岛屿个数&quot;&gt;&lt;/a&gt; 3513 岛屿个数&lt;/h1&gt;
&lt;p&gt;#外岛数量 #bfs&lt;/p&gt;
&lt;h2 id=&quot;杰克船长算法&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#杰克船长算法&quot;&gt;&lt;/a&gt; 杰克船长算法&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;杰克船长在公海上游荡，每发现一处岛屿，他就会绕着岛走一圈，并把这个岛标记到地图上。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;这个问题的解决方法就在这里：我们&lt;strong&gt;一定&lt;/strong&gt;要有一片完全连通的公海，只有在公海上遇到岛屿，才标记岛屿数量；绝不踏入内海。&lt;/p&gt;
&lt;p&gt;可是测试用例是这样的：&lt;/p&gt;
&lt;figure class=&quot;highlight plaintext&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;5 5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;01111&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11001&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10101&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10001&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11111&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;这个测试用例，只有&lt;code&gt;(0, 0)&lt;/code&gt;是公海，怎么办呢？&lt;br /&gt;
我们用一圈公海把测试用例包围起来：&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;private&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;static&lt;/span&gt; &lt;span class=&quot;keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;title function_&quot;&gt;processInput&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;(Scanner sc)&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    M = sc.nextInt();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    N = sc.nextInt();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    sc.nextLine();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    map = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;int&lt;/span&gt;[M + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;][N + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;]; &lt;span class=&quot;comment&quot;&gt;// 注意+2，多一圈&amp;#x27;0&amp;#x27;表示公海&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    visitedSea = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;boolean&lt;/span&gt;[M + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;][N + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;];&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    visitedIsland = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;boolean&lt;/span&gt;[M + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;][N + &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;];&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    cnt = &lt;span class=&quot;number&quot;&gt;0&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;x&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; x &amp;lt;= M; x += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;type&quot;&gt;String&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;line&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; sc.nextLine();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;y&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; y &amp;lt;= N; y += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            map[x][y] = line.charAt(y - &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) - &lt;span class=&quot;string&quot;&gt;&amp;#x27;0&amp;#x27;&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;</summary>
    
    
    
    
    <category term="笔记" scheme="http://simuleite.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
    
  </entry>
  
  <entry>
    <title>Spring核心</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Spring%E6%A0%B8%E5%BF%83/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/Spring%E6%A0%B8%E5%BF%83/</id>
    <published>2025-03-14T16:00:00.000Z</published>
    <updated>2025-05-11T08:53:23.195Z</updated>
    
    <content type="html"><![CDATA[<h1 id="spring核心思想"><a class="markdownIt-Anchor" href="#spring核心思想"></a> Spring核心思想</h1><p>Spring的核心是为Class创建代理对象实现一些AOP切面操作，从而支持方便的注解、事务、自动注入等功能。<br />为了创建代理对象，需要将对象创建移交给Spring完成，因此需要IoC容器。</p><h1 id="ioc"><a class="markdownIt-Anchor" href="#ioc"></a> IoC</h1><p>Inversion of Control<br />Spring通过控制反转，将对象创建交给IoC容器完成。<br />IoC容器实际上就是一个工厂，通过读取xml配置文件，使用反射创建对象。</p><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">bean</span> <span class="attr">id</span>=<span class="string">&quot;userDao&quot;</span> <span class="attr">class</span>=<span class="string">&quot;com.site.UserDao&quot;</span>&gt;</span><span class="tag">&lt;/<span class="name">bean</span>&gt;</span></span><br></pre></td></tr></table></figure><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">UserFactory</span> &#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="keyword">static</span> UserDao <span class="title function_">getDao</span><span class="params">()</span> &#123;</span><br><span class="line">        <span class="type">String</span> <span class="variable">classValue</span> <span class="operator">=</span> context.getProperty(<span class="string">&quot;userDao&quot;</span>);</span><br><span class="line">        <span class="type">Class</span> <span class="variable">clazz</span> <span class="operator">=</span> Class.forName(classValue);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>当我们的Dao文件路径改变时，只需要修改xml配置一处即可完成全部修改。<br />如果只用工厂模式，那需要导入很多包，也不直观。因此使用xml与反射，将工厂方法与配置解耦。</p><ul><li>BeanFactory：IoC容器基本使用，Spring内部使用<br />对象懒创建</li><li>ApplicationContext：BeanFactory子接口，暴露给开发者使用<br />加载配置就会创建对象</li></ul><h1 id="bean生命周期"><a class="markdownIt-Anchor" href="#bean生命周期"></a> Bean生命周期</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">类class -&gt; 无参构造方法 -&gt; 普通对象 -&gt; 依赖注入 -&gt; &quot;@PostConstruct&quot; -&gt; 初始化 -&gt; AOP -&gt; 代理对象 -&gt; Bean</span><br></pre></td></tr></table></figure><h1 id="spring-framework"><a class="markdownIt-Anchor" href="#spring-framework"></a> Spring Framework</h1><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">AnnotationConfigApplicationContext</span> <span class="variable">context</span> <span class="operator">=</span> </span><br><span class="line">    <span class="keyword">new</span> <span class="title class_">AnnotationConfigApplicationContext</span>(AppConfig.class);</span><br><span class="line"><span class="comment">// resource/application.xml</span></span><br><span class="line"><span class="type">ClassPathXmlApplicationContext</span> <span class="variable">context</span> <span class="operator">=</span> </span><br><span class="line">    <span class="keyword">new</span> <span class="title class_">ClassPathXmlApplicationContext</span>(<span class="string">&quot;application.xml&quot;</span>)</span><br><span class="line"></span><br><span class="line"><span class="type">UserService</span> <span class="variable">userService</span> <span class="operator">=</span> (UserService) context.getBean(<span class="string">&quot;userService&quot;</span>);</span><br><span class="line"></span><br></pre></td></tr></table></figure><h2 id="dependency-injection"><a class="markdownIt-Anchor" href="#dependency-injection"></a> Dependency Injection</h2><p>Spring首先是调用对象自身的构造方法创建对象，然后通过<strong>依赖注入</strong>（@Autowired属性赋值）来得到Bean</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">for</span> (Field field: userService.getClass().getFields()) &#123;</span><br><span class="line">    <span class="keyword">if</span> (field.isAnnotationPresent(Autowired.class)) &#123;</span><br><span class="line">        field.set(userService, value);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="postconstruct"><a class="markdownIt-Anchor" href="#postconstruct"></a> PostConstruct</h3><p>这个注解可以让Spring在初始化时调用此方法，从而实现一些初始化操作（如从数据库查询信息映射到实体类）。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@PostContruct</span></span><br><span class="line"><span class="keyword">public</span> <span class="title function_">init</span><span class="params">()</span> &#123;</span><br><span class="line">    ...</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> (Method method: userService.getClass().getMethods()) &#123;</span><br><span class="line">    <span class="keyword">if</span> (method.isAnnotationPresent(PostConstruct.class)) &#123;</span><br><span class="line">        method.invoke(userService, <span class="literal">null</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="aop"><a class="markdownIt-Anchor" href="#aop"></a> AOP</h3><p>AOP后，得到一个代理对象，然后Spring会在代理对象内部增加一个属性<code>UserService target</code>，并将经过依赖注入的普通对象赋值给target，然后调用<code>target.method()</code>，从而保留对象的所有Field的同时，可以通过代理在切面上做一些额外操作。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h1 id=&quot;spring核心思想&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#spring核心思想&quot;&gt;&lt;/a&gt; Spring核心思想&lt;/h1&gt;
&lt;p&gt;Spring的核心是为Class创建代理对象实现一些AOP切面操作，从而支持方便的注解、事务、自</summary>
      
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>JVM原理</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/JVM%E5%8E%9F%E7%90%86/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%9F%A5%E8%AF%86/JVM%E5%8E%9F%E7%90%86/</id>
    <published>2025-03-01T16:00:00.000Z</published>
    <updated>2025-04-25T14:32:31.642Z</updated>
    
    <content type="html"><![CDATA[<h1 id="程序如何装载"><a class="markdownIt-Anchor" href="#程序如何装载"></a> 程序如何装载</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Main\.java, Minor\.java -&gt; jar包.java Main\.main(): 编译打包</span><br><span class="line">jar包.java Main\.main() -&gt; 验证: 加载</span><br><span class="line">jar包.java Main\.main() -&gt; Minor\.class: 使用</span><br><span class="line">Minor\.class -&gt; JVM: 加载</span><br><span class="line">验证 -&gt; 准备 -&gt; 解析 -&gt; 初始化 -&gt; JVM</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>加载：从磁盘加载到内存。（懒加载，用到类才加载，如main方法或new对象）<br />验证：验证字节码是否正确、是否可识别。<br />准备：初始化静态（static，不包括常量）变量、赋初值（默认值）。<br />解析：符号引用 -&gt; 直接引用。静态方法（如main） -&gt; 指向数据所在内存的指针。这是静态链接，在类加载期间完成；而动态链接在程序运行期间完成。<br />初始化：为静态变量赋值，执行静态代码块。</p><h1 id="类加载器"><a class="markdownIt-Anchor" href="#类加载器"></a> 类加载器</h1><p>加载过程由类加载器实现，有几种类加载器：</p><ol><li>引导类加载器（C++）：JRE核心lib的jar类包</li><li>扩展类加载器：JRE拓展lib(ext)jar类包</li><li>应用程序类加载器：ClassPath路径下的类包（自己编写的类）</li><li>其他加载器：加载自定义路径下的类包</li></ol><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">java com\.site\.jvm\.Math\.class -&gt; java\.exe调用底层jvm\.dll创建Java虚拟机 -&gt; 创建引导类加载器实例</span><br><span class="line">创建引导类加载器实例 -&gt; sum\.misc\.Launcher\.getLauncher(): C++调用Java代码，创建JVM启动器实例，这个实例负责创建其他类加载器</span><br><span class="line">sum\.misc\.Launcher\.getLauncher() -&gt; launcher\.getClassLoader(): 获取运行类自己的加载器ClassLoader（AppClassLoader实例）</span><br><span class="line">launcher\.getClassLoader() -&gt; classLoader\.loadClass(&quot;com\.site\.jvm\.Math&quot;):调用loadClass加载即将要运行的类</span><br><span class="line">classLoader\.loadClass(&quot;com\.site\.jvm\.Math&quot;) -&gt; Math\.main(): 加载完成后，JVM执行Math.main()</span><br><span class="line">创建引导类加载器实例 -&gt; Math\.main(): C++发起调用 </span><br><span class="line">Math\.main()-&gt; JVM销毁: Java程序运行结束</span><br></pre></td></tr></table></figure><span id="more"></span><h1 id="双亲委派机制"><a class="markdownIt-Anchor" href="#双亲委派机制"></a> 双亲委派机制</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">direction: up</span><br><span class="line">应用程序类加载器 -&gt; 拓展类加载器: 向上委托</span><br><span class="line">拓展类加载器 -&gt; 引导类加载器: 向上委托</span><br><span class="line">引导类加载器 -&gt; 拓展类加载器: 父加载器加载失败，由子加载器自己加载</span><br><span class="line">拓展类加载器 -&gt; 应用程序类加载器: 父加载器加载失败，由子加载器自己加载</span><br></pre></td></tr></table></figure><p>pros:</p><ul><li>避免重复加载：下层加载了，上层不会加载；</li><li>沙箱安全机制：可以防止核心API被篡改<br />cons: 上层不能调动下层，层层传递比较繁琐</li></ul><h2 id="打破双亲委派-避免弊端"><a class="markdownIt-Anchor" href="#打破双亲委派-避免弊端"></a> 打破双亲委派 避免弊端</h2><p>通过<code>ContentTextClassLoader</code>反向委托，可以使上层调用你想要用的加载器。</p><h3 id="典型案例-tomcat8"><a class="markdownIt-Anchor" href="#典型案例-tomcat8"></a> 典型案例 Tomcat8</h3><h4 id="tomcat不遵循双亲委派机制自己写一个hashmap类会不会有风险"><a class="markdownIt-Anchor" href="#tomcat不遵循双亲委派机制自己写一个hashmap类会不会有风险"></a> Tomcat不遵循双亲委派机制，自己写一个HashMap类，会不会有风险？</h4><p>Tomcat不遵循双亲委派机制，只是自定义的classloader顺序不同，但是还是需要到顶层请求classloadder</p><h1 id="jvm内存布局"><a class="markdownIt-Anchor" href="#jvm内存布局"></a> JVM内存布局</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">grid-rows: 3</span><br><span class="line">cf: 类文件\nClass Files</span><br><span class="line">cf.shape: page</span><br><span class="line">cls: 类加载子系统\nClass Loader Subsystem</span><br><span class="line">rda: 运行时数据区（Runtime Data Area）: &#123;</span><br><span class="line">    方法区（共享）\nMethod Area</span><br><span class="line">    程序计数器\nPC Reg</span><br><span class="line">    本地方法栈\nNative Method Stack</span><br><span class="line">    堆（共享）\nHeap</span><br><span class="line">    虚拟机栈\nJVM Stack</span><br><span class="line">&#125;</span><br><span class="line">ee: 执行引擎\nExecution Engine</span><br><span class="line">nmi: 本地方法接口\nNative Method Interface</span><br><span class="line">nml: 本地方法库\nNative Method Libs</span><br><span class="line"></span><br><span class="line">cf &lt;-&gt; cls</span><br><span class="line">cls &lt;-&gt; rda</span><br><span class="line">rda &lt;-&gt; ee</span><br><span class="line">rda &lt;-&gt; nmi</span><br><span class="line">ee &lt;-&gt; nmi</span><br><span class="line">nmi &lt;-&gt; nml</span><br></pre></td></tr></table></figure><h1 id="垃圾回收"><a class="markdownIt-Anchor" href="#垃圾回收"></a> 垃圾回收</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">direction: right</span><br><span class="line">young: 年轻代: &#123;</span><br><span class="line">    ed: Eden(8)</span><br><span class="line">    s: Survivor区: &#123;</span><br><span class="line">        s0: s0(1)</span><br><span class="line">        s1: s1(1)</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line">young.ed -&gt; young.s.s0 &lt;-&gt; young.s.s1 -&gt; Old(2/3)</span><br></pre></td></tr></table></figure><p>当Eden区不够放，就会执行Minor GC，将空指针、无用对象回收，并把有用对象放入s0/s1（然后清楚Eden和另外一块survivor区的所有对象），年龄+1；<br />年龄到15时，会把对象放入老年代。<br />但是如果s0/s1放不下Minor GC后存活的对象，会直接放入老年代。<br />老年代满了，会触发Full GC，会暂停所有用户线程（STW， Stop The World）。</p><h1 id="对象创建"><a class="markdownIt-Anchor" href="#对象创建"></a> 对象创建</h1><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">start: 类加载检查</span><br><span class="line">cond: 是否已加载类</span><br><span class="line">cond.shape: diamond</span><br><span class="line">yes: 分配内存</span><br><span class="line">yes -&gt; 初始化 -&gt; 设置对象头 -&gt; 执行\&lt;init\&gt;方法</span><br><span class="line">no: 加载类</span><br><span class="line">start -&gt; cond</span><br><span class="line">cond -&gt; no: 否</span><br><span class="line">no -&gt; yes</span><br><span class="line">cond -&gt; yes: 是</span><br></pre></td></tr></table></figure><h2 id="分配内存"><a class="markdownIt-Anchor" href="#分配内存"></a> 分配内存</h2><ol><li>指针碰撞（默认方法）：Java堆中的内存规整分配，没有碎片，那么只需要在最后一个对象的指针后加上一段偏移量（大小）即可完成分配。</li><li>空闲列表：Java堆中内存分配不规整，碎片化。需要维护一张表，记录哪些空间可用。在分配内存时，找到足够大的空间划分给对象实例。</li></ol><blockquote><p>在并发的情况下，可能出现正在给A分配内存，指针未修改，此时又给B分配内存，B内存区与A重合的情况。</p></blockquote><h3 id="并发解决方法"><a class="markdownIt-Anchor" href="#并发解决方法"></a> 并发解决方法</h3><ol><li>CAS（Compare and Swap）：比较交换，虚拟机就采用CAS+失败重试的方法保证原子性。</li><li>TLAB（Thread Local Allocation Buffer）：按照不同线程划分内存区域，每个线程在区域内分配，防止重合。</li></ol><h2 id="object-header-对象头"><a class="markdownIt-Anchor" href="#object-header-对象头"></a> Object Header 对象头</h2><ol><li>Mark Word标记字段：运行时数据哈希值、GC分代年龄、锁状态标志、线程持有锁、偏向线程ID、偏向时间戳，32位4B，64位8B</li></ol><table><thead><tr><th>锁状态</th><th>23bit</th><th>2bit</th><th>4bit</th><th>1bit<br>是否指向偏向锁</th><th>2bit<br>锁标志位</th></tr></thead><tbody><tr><td>无锁态</td><td>对象的-</td><td>-HashCode</td><td>分代年龄</td><td>0</td><td>01</td></tr><tr><td>轻量级锁</td><td>指向-</td><td>-栈中锁-</td><td>-记录的-</td><td>-指针</td><td>00</td></tr><tr><td>重量级锁</td><td>指向-</td><td>-互斥量-</td><td>-（重量级锁）-</td><td>-的指针</td><td>10</td></tr><tr><td>GC标记</td><td>-</td><td>-</td><td>-</td><td>-</td><td>11</td></tr><tr><td>偏向锁</td><td>线程ID</td><td>Epoch</td><td>分代年龄</td><td>1</td><td>01</td></tr></tbody></table><ol start="2"><li>Klass Pointer类型指针：指向类的元数据，8B，压缩后4B</li><li>数组长度：4B</li></ol><h2 id="init-方法"><a class="markdownIt-Anchor" href="#init-方法"></a> &lt;init&gt; 方法</h2><p>执行&lt;init&gt;方法，会将对象按照程序的意愿进行初始化，是真正的属性赋值（不是赋初值0），会执行构造方法。</p><h2 id="对象内存分配"><a class="markdownIt-Anchor" href="#对象内存分配"></a> 对象内存分配</h2><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">start -&gt; cond1: new Object()</span><br><span class="line">cond1: 栈内分配？</span><br><span class="line">cond1 -&gt; 栈: Y</span><br><span class="line">栈 -&gt; End: POP</span><br><span class="line">cond1 -&gt; cond2: N</span><br><span class="line">cond2: 大对象？</span><br><span class="line">cond2 -&gt; Old: Y</span><br><span class="line">Old -&gt; End: Full GC</span><br><span class="line">cond2 -&gt; cond3: N</span><br><span class="line">cond3: TLAB？</span><br><span class="line">cond3 -&gt; Eden: Y</span><br><span class="line">cond3 -&gt; Eden: N</span><br><span class="line">condMgc: Minor GC?</span><br><span class="line">Eden -&gt; condMgc</span><br><span class="line">condMgc -&gt; S1: N</span><br><span class="line">S1 -&gt; Age?</span><br><span class="line">Age? -&gt; Old: Y</span><br><span class="line">Age? -&gt; S2: N</span><br><span class="line">S2 -&gt; condMgc</span><br><span class="line">condMgc &lt;-&gt; End: Y</span><br></pre></td></tr></table></figure><h3 id="对象逃逸分析"><a class="markdownIt-Anchor" href="#对象逃逸分析"></a> 对象逃逸分析</h3><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> Object <span class="title function_">method1</span><span class="params">()</span> &#123;</span><br><span class="line">    <span class="type">Object</span> <span class="variable">obj</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Object</span>();</span><br><span class="line">    ...</span><br><span class="line">    <span class="keyword">return</span> Object; <span class="comment">// 被其他方法使用，逃逸</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">method2</span><span class="params">()</span> &#123;</span><br><span class="line">    <span class="type">Object</span> <span class="variable">obj</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">Object</span>();</span><br><span class="line">    ...</span><br><span class="line">    businessMapper.insert(obj); <span class="comment">// 生命周期和函数一起结束，非逃逸</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>非逃逸对象会分配在栈空间，随着方法结束被释放；而逃逸对象会真正分配在堆上。</p><h2 id="survivor到老年代"><a class="markdownIt-Anchor" href="#survivor到老年代"></a> Survivor到老年代</h2><p>一块Survivor区，一批对象占用&lt;=50%的内存大小。如果一批对象大于这个值，那么大于等于这批对象最大年龄的对象都会被放入老年代。</p><p>因此，如果一秒时间内新对象太多，超出这个Survivor区的生存阈值，就会直接放入老年代，从到导致更频繁的Full GC。解决方法是：1. 要么扩大Eden，让Minor GC间隔更长；2. 要么扩大Survivor，让垃圾对象能被放入，并在下一次GC及时释放，不会被错误地放入老年代。</p><h2 id="full-gc"><a class="markdownIt-Anchor" href="#full-gc"></a> Full GC</h2><p>每次Minor GC，都会判断老年代内存空间是否够Survivor区的对象放入老年代，不够，则触发Full GC，还不够，则OOM<br />当老年代空间不够放，JVM会尽快Full GC（因为即使Minor GC可能也不够放，反而浪费了时间）</p><h2 id="对象内存回收算法"><a class="markdownIt-Anchor" href="#对象内存回收算法"></a> 对象内存回收算法</h2><h3 id="1-引用计数器"><a class="markdownIt-Anchor" href="#1-引用计数器"></a> 1. 引用计数器</h3><p>新增一个引用，计数器加1；一个引用失效，计数器减1。<br />实现简单，但是存在循环引用问题，A&lt;-&gt;B之间互相引用，于是无法回收。</p><h3 id="2-可达性分析"><a class="markdownIt-Anchor" href="#2-可达性分析"></a> 2. 可达性分析</h3><p>以GC Roots对象（线程栈本地对象、静态变量、本地方法栈变量等）作为起点，向下搜索引用对象，找到的对象都标记为非垃圾对象。</p><h2 id="无用类判断"><a class="markdownIt-Anchor" href="#无用类判断"></a> 无用类判断</h2><p>同时满足以下3个条件即为无用类：</p><ol><li>所有实例都已被回收</li><li>该类的ClassLoader已被回收</li><li>该类的java.lang.Class对象没有被引用。<br />一般只有自定义的类加载器会被回收。</li></ol><h1 id="垃圾收集"><a class="markdownIt-Anchor" href="#垃圾收集"></a> 垃圾收集</h1><h2 id="垃圾收集算法"><a class="markdownIt-Anchor" href="#垃圾收集算法"></a> 垃圾收集算法</h2><h3 id="分代收集理论"><a class="markdownIt-Anchor" href="#分代收集理论"></a> 分代收集理论</h3><p>根据对象存活周期的不同，将内存分成几块（在不同的年龄代，用不同的垃圾收集算法）。一般Java堆分为新生代和老年代。<br />例如，新生代几乎99%的对象会被回收，所以简单地标记-复制就可以完成内存整理；而老年代存活几率比较高，而且可能没有额外空间分配，因此需要用标记-清除、标记-整理算法。</p><blockquote><p>标记-复制比标记-清除和标记-整理快10倍以上。</p></blockquote><h3 id="标记-复制算法"><a class="markdownIt-Anchor" href="#标记-复制算法"></a> 标记-复制算法</h3><p>这个算法把内存分为大小相等的两块，每次整理只需要将一端的存活对象标记好，复制到另一端，然后把这一端内存全部清空。因此，使用标记-复制算法，每次都对内存区间的一半进行回收。（内存利用率低，最多只有50%）</p><h3 id="标记-清除算法"><a class="markdownIt-Anchor" href="#标记-清除算法"></a> 标记-清除算法</h3><p>这个算法简单地标记存活对象，然后一次性清除未被标记的对象。算法实现简单，但是也有问题：</p><ol><li>效率低。如果标记的对象太多而且内存不连续，效率不高。</li><li>空间碎片化。只是简单地清除对象而不整理，会产生大量内存碎片，缺少整块内存。</li></ol><h3 id="标记-整理算法"><a class="markdownIt-Anchor" href="#标记-整理算法"></a> 标记-整理算法</h3><p>这个算法标记存活对象，然后让所有存活对象都向内存一端移动。然后清除其余的内存区间，获得整块连续的内存空间。</p><h2 id="垃圾收集器"><a class="markdownIt-Anchor" href="#垃圾收集器"></a> 垃圾收集器</h2><blockquote><p>垃圾收集器是内存回收的具体实现。</p></blockquote><h3 id="1-serial收集器"><a class="markdownIt-Anchor" href="#1-serial收集器"></a> 1 Serial收集器</h3><p>串行收集器是最基础的收集器。它的单线程体现在：</p><ol><li>只用一条垃圾收集线程</li><li>垃圾收集时，Stop The World<br />Serial在新生代<code>-XX:+UserSerialGC</code>使用标记-复制算法，在老年代<code>-XX:+UserSerialOldGC</code>使用标记-整理算法。</li></ol><h3 id="2-parallel-scavenge收集器jdk18默认收集器"><a class="markdownIt-Anchor" href="#2-parallel-scavenge收集器jdk18默认收集器"></a> 2 Parallel Scavenge收集器（JDK1.8默认收集器）</h3><p>并行收集器，实际上就是Serial的多线程版本，回收时同样会Stop The World。默认的收集线程数与CPU核心数量保持一致。<br />并行收集器关注CPU吞吐量，减少用户线程停顿时间。<br />Parallel Scavenge同样在新生代<code>-XX:+UserParallelGC</code>使用标记-复制算法，在老年代<code>-XX:+UserParallelOldGC</code>使用标记-整理算法。</p><h4 id="parnew收集器"><a class="markdownIt-Anchor" href="#parnew收集器"></a> ParNew收集器</h4><p>ParNew收集器<code>-XX:+UseParNewGC</code>和Parallel类似，但是可以与CMS配合使用。</p><h3 id="3-cms收集器"><a class="markdownIt-Anchor" href="#3-cms收集器"></a> 3 CMS收集器</h3><p>CMS（Concurrent Mark Sweep）收集器只适用于老年代，以<strong>最短停顿时间</strong>为目标。是HotSpot JVM第一款并发收集器，可以使垃圾回收与用户线程同时工作。<br />CMS使用的是标记-清除算法（Mark Sweep），运作过程是：</p><ol><li>初始标记：STW，记录GC Roots直接引用的对象。</li><li>并发标记：从GC Roots直接关联对象开始遍历整个对象图，过程中不需要暂停用户线程。因为没有STW，所以过程中已经标记的对象状态会改变。</li><li>重新标记：STW，用三色标记的增量更新算法做重新标记，修正第2步状态改变的对象标记记录。</li><li>并发清理：恢复用户线程，同时开始GC清扫。</li><li>并发重置：重置本次GC的标记数据。<br />CMS主要优点是并发收集、短停顿，然而也有下面几个缺点：</li><li>对CPU资源敏感，会和服务器抢资源。</li><li>无法处理<strong>浮动垃圾</strong>（并发标记、并发清理阶段产生的新垃圾），只能等下一次GC再清理。</li><li>清扫算法，不能腾出整块连续内存，只能得到许多内存碎片。</li><li>执行不确定性。在并发标记、并发清理阶段会出现上一轮垃圾回收还没完成，下一轮又开始的情况。</li></ol><h4 id="并发失败"><a class="markdownIt-Anchor" href="#并发失败"></a> 并发失败</h4><p>在CMS并发标记、并发清理阶段，如果用户线程又实例化了许多新对象，导致老年代触发Full GC，STW，那么这个清理线程实际上就失败了，还是用效率低下的线性收集器收集。</p><h4 id="cms参数"><a class="markdownIt-Anchor" href="#cms参数"></a> CMS参数</h4><p><code>-XX:+UseConcMarkSweepGC</code> 启用CMS（老年代）<br /><code>-XX:ConcGCThreads</code> 并发GC线程数量<br /><code>-XX:+UseCMSCompactAtFullCollection</code> FullGC后碎片整理<br /><code>-XX:CMSFullGCsBeforeCompaction</code> 设置每隔多少次FullGC做一次碎片整理，默认值为0<br /><code>-XX:CMSInitiatingOccupancyFraction</code> 设置老年代FullGC空间占比阈值，默认是92；不设置100是为了避免并发失败启用线性收集器<br /><code>-XX:+UseCMSInitiatingOccupancyOnly</code> 强制使用设定阈值。默认是只在第一次使用设定的阈值，后续动态调整<br /><code>-XX:+CMSScavengeBeforeRemark</code> CMSGC前启动一次MinorGC，降低CMS标记阶段的开销<br /><code>-XX:+CMSParallelInitialMarkEnabled</code> 初始标记阶段使用多线程，缩短STW<br /><code>-XX:+CMSParallelRemarkEnabled</code> 重新标记阶段采用多线程，缩短STW</p><h2 id="三色标记"><a class="markdownIt-Anchor" href="#三色标记"></a> 三色标记</h2><p>黑色：全部引用都扫描过的对象，存活<br />灰色：还没完全扫描的对象<br />白色：未扫描的对象。扫描开始时所有对象为白色；扫描结束后，清理白色对象<br />三色标记可能会产生漏标的问题，对于灰色对象中还没扫描的对象，如果这个对象被已经扫描过的黑色对象引用，而灰色对象的引用又被置null，那么这个应该被扫描和标记的对象就会被漏扫，从而导致错误清理。</p><h3 id="漏标-读写屏障"><a class="markdownIt-Anchor" href="#漏标-读写屏障"></a> 漏标-读写屏障</h3><p>漏标会错误清理，是非常严重的错误，会使用增量更新和原始快照两种方法避免：</p><ul><li>增量更新（IU）：当黑色对象新增白色对象引用时，会把这个新增引用记录下来（这样黑色就变成灰色对象），等并发扫描结束，重新扫描这些黑色对象。</li><li>原始快照（STAB）：当灰色对象删除白色对象的引用时，记录这个引用删除，等并发扫描结束，将这些白色对象直接设置为黑色对象，使这一轮GC不清理这些对象。</li></ul><h4 id="写屏障"><a class="markdownIt-Anchor" href="#写屏障"></a> 写屏障</h4><figure class="highlight c"><table><tr><td class="code"><pre><span class="line"><span class="type">void</span> <span class="title function_">oop_field_store</span><span class="params">(oop* field, oop new_value)</span> &#123;</span><br><span class="line">    pre_write_barrier(field); <span class="comment">// 写屏障，记录旧值（先加入队列，与实际操作异步）</span></span><br><span class="line">    *field = new_value; <span class="comment">// 赋值操作</span></span><br><span class="line">    post_write_barrier(field, value); <span class="comment">// 写屏障，记录新值（入队，异步执行）</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>CMS使用的是写屏障+增量更新方案<br />G1，Shenandoah使用写屏障+STAB方案<br />ZGC使用读屏障方案</p><h4 id="为什么g1使用stabcms用增量更新"><a class="markdownIt-Anchor" href="#为什么g1使用stabcms用增量更新"></a> 为什么G1使用STAB，CMS用增量更新？</h4><p>不处理，效率更高。有多少被删除的引用会真的被黑色对象引用呢？再做一次深度扫描太浪费了。而G1是为大内存设计的，分了很多区域，与只有一块老年代区域的CMS不同，做深度扫描的成本会高很多。</p><h2 id="记忆集与卡表"><a class="markdownIt-Anchor" href="#记忆集与卡表"></a> 记忆集与卡表</h2><p>在新生代做GC Roots可达性扫描过程中，可能会碰到跨代引用的问题（如被老年代引用）。这时，如果要进入老年代做扫描，效率太低了。<br />因此，在新生代引入记录集（Remember Set）数据结构，记录从非收集区到收集区的指针集合，避免把整个老年代纳入GCRoots扫描范围。</p><p>Hotspot使用一种叫“卡表”（Cardtable）的方式实现记忆集。<br />卡表使用一个字节数组<code>CARD_TABLE[]</code>实现，其中每个元素对应一块特定大小的内存，称为“卡页”。<br />Hotspot使用的卡页大小为<code>2^9 = 512</code>字节。即<code>CARD_TABLE</code>1字节对应512B（0~511）</p><p>具体来说，老年代按卡页被分割成许多个区块。当老年代地址<code>0x0200（2*16^2=512B）</code>区块引用了新生代的对象，<code>CARD_TABLE[] = &#123; 0, 1, ... &#125;</code>，将对应区域标记为dirty。此时，新生代不仅会做GC Roots扫描，还会到老年代对应的地址<code>0x0200</code>扫描引用。</p><p>卡表状态也是用写屏障维护。</p><h2 id="g1收集器"><a class="markdownIt-Anchor" href="#g1收集器"></a> G1收集器</h2><p>使用<code>-XX:+UseG1GC</code>启动G1（Garbage-First）。<br />G1不再使用原来的分代概念，而是将内存分割成大小相等的区域（Region）。JVM最多可以有2048个Region（每块区域大小=堆大小/2048）。<br />G1保留了年轻代、老年代的概念，但是它们都可以随机放在任何一块Region中，而不是物理隔离。<br />默认的年轻代占堆空间的5%，在运行过程中，JVM会给年轻代增加内存，但是年轻代不会超过60%。在年轻代内部，仍然保持8：1：1的比例</p><p>G1的特点是有一个<code>Humongous（极大的）</code>分区，专门存放短期巨型对象（超过region50%就算大对象，如果超过一个region大小，那就跨region存放），而不是直接放入老年代。在FullGC时，会将Humongous区垃圾对象一并回收。</p><h3 id="gc过程"><a class="markdownIt-Anchor" href="#gc过程"></a> GC过程</h3><ol><li>初始标记，STW</li><li>并发标记，和CMS一样</li><li>最终标记：和CMS重新标记一样</li><li>筛选回收：对各个Region的回收价值/成本进行排序，根据用户期望停顿时间<code>-XX:MaxGCPauseMillis，默认200ms</code>制定回收计划。</li></ol><blockquote><p>因为是用户设置的停顿时间，所以G1直接STW并发回收提高效率。</p></blockquote><p>G1主要使用复制算法，将一个region的存活对象放入另一个region中，内存碎片比较少，而且不像CMS需要整理。<br />G1因为内部实现复杂，没用实现并发回收。Shenandoah就实现了这一点，可以看作G1的升级版。<br />G1在后台维护一个优先队列，优先选择允许时间内价值最大（回收空间最多）的Region</p><h3 id="g1垃圾收集分类"><a class="markdownIt-Anchor" href="#g1垃圾收集分类"></a> G1垃圾收集分类</h3><h4 id="younggc"><a class="markdownIt-Anchor" href="#younggc"></a> YoungGC</h4><p>YoungGC会计算Eden区回收所需的时间，接近用户设置的允许时间时，就会触发YoungGC。<br />G1通过动态调整Eden区大小（默认5%）来实现上述算法。</p><h4 id="mixedgc"><a class="markdownIt-Anchor" href="#mixedgc"></a> MixedGC</h4><p>老年代对占有率达到<code>-XX:InitiatingHeapOccupancyPercent，默认45</code>的设定值就会触发，将回收所有Young、部分Old和Humongous区域。<br />G1会优先在老年代做MixedGC，如果复制对象过程中，没有足够的内存，那么会触发FullGC。</p><h4 id="fullgc"><a class="markdownIt-Anchor" href="#fullgc"></a> FullGC</h4><p>STW，线性标记-压缩，整个过程比较耗时；Shenandoah优化这个过程为多线程。</p><h2 id="zgc"><a class="markdownIt-Anchor" href="#zgc"></a> ZGC</h2><p>ZGC的设计目标：</p><ol><li>TB级别的堆内存</li><li>GC Pause &lt;= 10ms</li><li>下一代GC特征基础</li><li>最多15%的吞吐量下降<br />ZGC源自Azul的C4，最大优势是，停顿时间与堆大小<strong>无关</strong>，而是都在10ms内。</li></ol><p>NUMA架构，识别每块CPU使用的内存区域，防止竞争和锁的效率问题</p><h3 id="zgc运作过程"><a class="markdownIt-Anchor" href="#zgc运作过程"></a> ZGC运作过程</h3><ol><li>并发标记：ZGC不把标记放在对象内部，而是在颜色指针上标记</li><li>并发预备重分配：得出本次收集需要清理的Region并放入重分配集Relocation Set，与G1不同，ZGC每次都会扫描所有Region，这样就不需要维护卡表</li><li>并发重分配：会将Relocation Set的存活对象复制到新的Region上。由于会发生数据不同步问题。因此维护一个转发表（Forward Table），并通过读屏障来确保数据一致，ZGC称之为Self-Healing。</li><li>并发重映射：修正整个堆指向重分配集中，旧对象的所有引用</li></ol><h3 id="读屏障懒更新"><a class="markdownIt-Anchor" href="#读屏障懒更新"></a> 读屏障（懒更新）</h3><p>由于ZGC使用颜色指针，而复制对象的过程中会发生数据不同步问题。<br />所以，ZGC直到原来的指针被读取（即此时没有发生写入），才会真正地修正指针引用，成为读屏障。</p><h3 id="转发表"><a class="markdownIt-Anchor" href="#转发表"></a> 转发表</h3><p>读屏障怎么知道地址有没有变化？在并发重分配阶段维护转发表，就知道对象去向。</p><h3 id="颜色指针"><a class="markdownIt-Anchor" href="#颜色指针"></a> 颜色指针</h3><p>以前的垃圾回收器GC信息保存在对象头，而ZGC将这些信息保存在指针上。<br />每个对象有一个64位的指针，其中</p><ul><li>42位用于寻址（4^42=4T）</li><li>1位Marked1标识</li><li>1位Marked0标识</li><li>1位Remapped标识，设置后，说明对象没有指向RelocationSet</li><li>1位Finalizable标识，与并发引用处理有关，表示这个对象只能通过finalizer访问</li><li>18位未使用</li></ul><h4 id="为什么2个mark"><a class="markdownIt-Anchor" href="#为什么2个mark"></a> 为什么2个Mark</h4><p>每个GC周期开始，会交替使用标记位（01、10互换），使上次GC标记失效。</p><h4 id="颜色指针3大优势"><a class="markdownIt-Anchor" href="#颜色指针3大优势"></a> 颜色指针3大优势</h4><ol><li>一旦某个Region的存活对象被移走，这个Region能够立刻被释放和重用。</li><li>颜色指针大幅减少内存屏障使用数量，ZGC使用读屏障</li><li>颜色指针有强大扩展性（18位）</li></ol><h3 id="zgc问题"><a class="markdownIt-Anchor" href="#zgc问题"></a> ZGC问题</h3><p>最大的问题也是浮动垃圾，ZGC的停顿时间是10ms，但是实际上执行回收时间远大于这个值，在此期间会产生许许多多不能处理的新垃圾对象，只能等待下一次回收。</p><p>解决方法只有增加堆容量，让程序有更多喘息时间（未分代情况下）。<br />分代ZGC区分新生代和老年代（代际隔离），对新生代的回收更频繁（分代回收）。</p><h1 id="jvm调优工具"><a class="markdownIt-Anchor" href="#jvm调优工具"></a> JVM调优工具</h1><h2 id="jmap"><a class="markdownIt-Anchor" href="#jmap"></a> Jmap</h2><p><code>jmap</code>可以查看内存信息、实例个数以及所占内存大小</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jps</span></span><br><span class="line">&lt;PID&gt; &lt;Name&gt;</span><br><span class="line">22928 Jps</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jmap -histo &lt;PID&gt; &gt; jlog</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">查看堆情况</span></span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jmap -heap &lt;PID&gt;</span> </span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">导出快照Dump</span></span><br><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jmap -dump:format=b,file=&lt;FileName&gt; &lt;PID&gt;</span></span><br></pre></td></tr></table></figure><h2 id="jstack"><a class="markdownIt-Anchor" href="#jstack"></a> Jstack</h2><p><code>jstack</code>可以查找死锁</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jstack &lt;PID&gt;</span></span><br><span class="line">...</span><br><span class="line">java.lang.Thread.State: BLOCKED</span><br><span class="line">...</span><br><span class="line">Found &lt;amount&gt; Java-level deadlock:</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h2 id="jvisualvm"><a class="markdownIt-Anchor" href="#jvisualvm"></a> Jvisualvm</h2><p><code>jvisualvm</code>可视化监管java进程，点击<code>进程dump</code>与<code>jstack</code>一致</p><h2 id="jstat"><a class="markdownIt-Anchor" href="#jstat"></a> Jstat</h2><p><code>jstat [-option] [vmid] [gap(ms)] [query_times]</code>可以查看堆内存各部分使用情况，例如查看GC：</p><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">jstat -gc &lt;PID&gt;</span></span><br></pre></td></tr></table></figure><p>结合不同参数可以具体查看各代的情况。</p><h2 id="linux工具"><a class="markdownIt-Anchor" href="#linux工具"></a> Linux工具</h2><h3 id="top"><a class="markdownIt-Anchor" href="#top"></a> top</h3><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">$ </span><span class="language-bash">top -p &lt;PID&gt;</span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">H查看所有线程情况，配合jstack找到线程ID，即可找到对应代码</span></span><br></pre></td></tr></table></figure><h1 id="实用脚本"><a class="markdownIt-Anchor" href="#实用脚本"></a> 实用脚本</h1><h2 id="jstat-2"><a class="markdownIt-Anchor" href="#jstat-2"></a> Jstat</h2><figure class="highlight shell"><table><tr><td class="code"><pre><span class="line"><span class="meta prompt_">#</span><span class="language-bash">!/bin/bash</span></span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">检查是否提供了PID参数</span></span><br><span class="line">if [ -z &quot;$1&quot; ]; then</span><br><span class="line">  echo &quot;Usage: $0 &lt;PID&gt;&quot;</span><br><span class="line">  echo &quot;Example: $0 5527&quot;</span><br><span class="line">  exit 1</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line">PID=$1</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">检查进程是否存在</span></span><br><span class="line">if ! ps -p $PID &gt; /dev/null; then</span><br><span class="line">  echo &quot;Error: Process with PID $PID not found&quot;</span><br><span class="line">  exit 1</span><br><span class="line">fi</span><br><span class="line"><span class="meta prompt_"></span></span><br><span class="line"><span class="meta prompt_"># </span><span class="language-bash">执行jstat并格式化输出</span></span><br><span class="line">jstat -gc $PID | awk &#x27;</span><br><span class="line">function progress(pct) &#123;</span><br><span class="line">    bars = int(pct/5)</span><br><span class="line">    return sprintf(&quot;[%-20s]&quot;, substr(&quot;||||||||||||||||||||&quot;, 1, bars))</span><br><span class="line">&#125;</span><br><span class="line">NR==2 &#123;</span><br><span class="line">    printf &quot;Eden:  %5.1fMB %s %5.1fMB (%d%%)\n&quot;, $6/1024, progress(100*$6/$5), $5/1024, 100*$6/$5</span><br><span class="line">    printf &quot;Old:   %5.1fMB %s %5.1fMB (%d%%)\n&quot;, $8/1024, progress(100*$8/$7), $7/1024, 100*$8/$7</span><br><span class="line">    printf &quot;Meta:  %5.1fMB %s %5.1fMB (%d%%)\n&quot;, $10/1024, progress(100*$10/$9), $9/1024, 100*$10/$9</span><br><span class="line">    printf &quot;GC Stats: YGC=%d(%.3fs) FGC=%d(%.3fs) Total=%.3fs\n&quot;, $13, $14, $15, $16, $19</span><br><span class="line">&#125;&#x27;</span><br><span class="line"></span><br><span class="line">exit 0</span><br></pre></td></tr></table></figure><h3 id="输出"><a class="markdownIt-Anchor" href="#输出"></a> 输出</h3><figure class="highlight plaintext"><table><tr><td class="code"><pre><span class="line">Eden:   32.0MB [|||||||||||||       ]  47.0MB (68%)</span><br><span class="line">Old:    18.8MB [||||||||||||        ]  30.0MB (62%)</span><br><span class="line">Meta:   52.5MB [||||||||||||||||||| ]  52.9MB (99%)</span><br><span class="line">GC Stats: YGC=18(0.166s) FGC=0(0.000s) Total=0.174s</span><br></pre></td></tr></table></figure>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;程序如何装载&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#程序如何装载&quot;&gt;&lt;/a&gt; 程序如何装载&lt;/h1&gt;
&lt;figure class=&quot;highlight plaintext&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;Main&#92;.java, Minor&#92;.java -&amp;gt; jar包.java Main&#92;.main(): 编译打包&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;jar包.java Main&#92;.main() -&amp;gt; 验证: 加载&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;jar包.java Main&#92;.main() -&amp;gt; Minor&#92;.class: 使用&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;Minor&#92;.class -&amp;gt; JVM: 加载&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;验证 -&amp;gt; 准备 -&amp;gt; 解析 -&amp;gt; 初始化 -&amp;gt; JVM&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;加载：从磁盘加载到内存。（懒加载，用到类才加载，如main方法或new对象）&lt;br /&gt;
验证：验证字节码是否正确、是否可识别。&lt;br /&gt;
准备：初始化静态（static，不包括常量）变量、赋初值（默认值）。&lt;br /&gt;
解析：符号引用 -&amp;gt; 直接引用。静态方法（如main） -&amp;gt; 指向数据所在内存的指针。这是静态链接，在类加载期间完成；而动态链接在程序运行期间完成。&lt;br /&gt;
初始化：为静态变量赋值，执行静态代码块。&lt;/p&gt;
&lt;h1 id=&quot;类加载器&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#类加载器&quot;&gt;&lt;/a&gt; 类加载器&lt;/h1&gt;
&lt;p&gt;加载过程由类加载器实现，有几种类加载器：&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;引导类加载器（C++）：JRE核心lib的jar类包&lt;/li&gt;
&lt;li&gt;扩展类加载器：JRE拓展lib(ext)jar类包&lt;/li&gt;
&lt;li&gt;应用程序类加载器：ClassPath路径下的类包（自己编写的类）&lt;/li&gt;
&lt;li&gt;其他加载器：加载自定义路径下的类包&lt;/li&gt;
&lt;/ol&gt;
&lt;figure class=&quot;highlight plaintext&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;java com&#92;.site&#92;.jvm&#92;.Math&#92;.class -&amp;gt; java&#92;.exe调用底层jvm&#92;.dll创建Java虚拟机 -&amp;gt; 创建引导类加载器实例&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;创建引导类加载器实例 -&amp;gt; sum&#92;.misc&#92;.Launcher&#92;.getLauncher(): C++调用Java代码，创建JVM启动器实例，这个实例负责创建其他类加载器&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;sum&#92;.misc&#92;.Launcher&#92;.getLauncher() -&amp;gt; launcher&#92;.getClassLoader(): 获取运行类自己的加载器ClassLoader（AppClassLoader实例）&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;launcher&#92;.getClassLoader() -&amp;gt; classLoader&#92;.loadClass(&amp;quot;com&#92;.site&#92;.jvm&#92;.Math&amp;quot;):调用loadClass加载即将要运行的类&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;classLoader&#92;.loadClass(&amp;quot;com&#92;.site&#92;.jvm&#92;.Math&amp;quot;) -&amp;gt; Math&#92;.main(): 加载完成后，JVM执行Math.main()&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;创建引导类加载器实例 -&amp;gt; Math&#92;.main(): C++发起调用 &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;Math&#92;.main()-&amp;gt; JVM销毁: Java程序运行结束&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;</summary>
    
    
    
    
    <category term="知识" scheme="http://simuleite.github.io/tags/%E7%9F%A5%E8%AF%86/"/>
    
  </entry>
  
  <entry>
    <title>洛谷 错题本</title>
    <link href="http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/%E9%94%99%E9%A2%98%E6%9C%AC/%E6%B4%9B%E8%B0%B7%20%E9%94%99%E9%A2%98%E6%9C%AC/"/>
    <id>http://simuleite.github.io/ComputerScience/%E7%AC%94%E8%AE%B0/%E9%94%99%E9%A2%98%E6%9C%AC/%E6%B4%9B%E8%B0%B7%20%E9%94%99%E9%A2%98%E6%9C%AC/</id>
    <published>2025-02-28T16:00:00.000Z</published>
    <updated>2025-03-12T01:32:05.118Z</updated>
    
    <content type="html"><![CDATA[<h1 id="p1004-noip-2000-提高组-方格取数"><a class="markdownIt-Anchor" href="#p1004-noip-2000-提高组-方格取数"></a> P1004 [NOIP 2000 提高组] 方格取数</h1><p>#走两次dp<br />如果只走一次，这题是非常经典的DP。但是要走两次，就变得非常有难度。<br />首先，可以简单地推广：要走两次，dp就存四个下标：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span>[][][][] dp = <span class="keyword">new</span> <span class="title class_">int</span>[N][N][N][N];</span><br></pre></td></tr></table></figure><p>我们只需要遍历所有可能，并且比较四种走法（同下、同右、一下一右），取最大值就可以了。<br />注意，一个数只能取一次，需要一个判断防止重复取数。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i1</span> <span class="operator">=</span> <span class="number">1</span>; i1 &lt; N; i1 += <span class="number">1</span>) &#123;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i2</span> <span class="operator">=</span> <span class="number">1</span>; i2 &lt; N; i2 += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j1</span> <span class="operator">=</span> <span class="number">1</span>; j1 &lt; N; j1 += <span class="number">1</span>) &#123;</span><br><span class="line">            <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j2</span> <span class="operator">=</span> <span class="number">1</span>; j2 &lt; N; j2 += <span class="number">1</span>) &#123;</span><br><span class="line">                <span class="type">int</span> <span class="variable">step</span> <span class="operator">=</span> map[i1][j1];</span><br><span class="line">                <span class="keyword">if</span> (i2 != i1 &amp;&amp; j2 != j1) step += map[i2][j2];</span><br><span class="line">                </span><br><span class="line">                dp[i1][j1][i2][j2] = </span><br><span class="line">                    Math.max(dp[i1-<span class="number">1</span>][j1][i2-<span class="number">1</span>][j2], </span><br><span class="line">                    Math.max(dp[i1-<span class="number">1</span>][j1][i2][j2-<span class="number">1</span>], </span><br><span class="line">                    Math.max(dp[i1][j1-<span class="number">1</span>][i2-<span class="number">1</span>][j2], </span><br><span class="line">                             dp[i1][j1-<span class="number">1</span>][i2][j2-<span class="number">1</span>]))) + step;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line">System.out.println(dp[N-<span class="number">1</span>][N-<span class="number">1</span>][N-<span class="number">1</span>][N-<span class="number">1</span>]);</span><br></pre></td></tr></table></figure><p>当然，4个循环时间复杂度太高了。我们可以用一个<code>k == i1 + j1 == i2 + j2</code>来减少一重循环。<br />这个k利用得很巧妙，因为每次要么向下走，要么向右走，所以<code>k-1 == i-1 + j == i + j-1</code>，全程使用<code>k-1</code>就能代表所有情况。</p><span id="more"></span><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span>[][][] dp = <span class="keyword">new</span> <span class="title class_">int</span>[<span class="number">2</span>*N][N][N];</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">k</span> <span class="operator">=</span> <span class="number">1</span>; k &lt; <span class="number">2</span>*N; k += <span class="number">1</span>) &#123;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i1</span> <span class="operator">=</span> <span class="number">1</span>; i1 &lt; N; i1 += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i2</span> <span class="operator">=</span> <span class="number">1</span>; i2 &lt; N; i2 += <span class="number">1</span>) &#123;</span><br><span class="line">            <span class="type">int</span> <span class="variable">j1</span> <span class="operator">=</span> k - i1, j2 = k - i2;</span><br><span class="line">            <span class="keyword">if</span> (j1 &lt; <span class="number">0</span> || j1 &gt;= N || j2 &lt; <span class="number">0</span> || j2 &gt;= N) <span class="keyword">continue</span>;</span><br><span class="line"></span><br><span class="line">            <span class="type">int</span> <span class="variable">step</span> <span class="operator">=</span> map[i1][j1];</span><br><span class="line">            <span class="keyword">if</span> (i1 != i2) step += map[i2][j2];</span><br><span class="line">            dp[k][i1][i2] = Math.max(dp[k-<span class="number">1</span>][i1-<span class="number">1</span>][i2-<span class="number">1</span>], Math.max(dp[k-<span class="number">1</span>][i1][i2],</span><br><span class="line">            Math.max(dp[k-<span class="number">1</span>][i1-<span class="number">1</span>][i2], dp[k-<span class="number">1</span>][i1][i2-<span class="number">1</span>]))) + step;</span><br><span class="line"></span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line">System.out.println(dp[<span class="number">2</span> * (N-<span class="number">1</span>)][N-<span class="number">1</span>][N-<span class="number">1</span>]);</span><br></pre></td></tr></table></figure><h1 id="b3637-最长上升子序列"><a class="markdownIt-Anchor" href="#b3637-最长上升子序列"></a> B3637 最长上升子序列</h1><p>#单维dp</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span>[] dp = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line"><span class="type">int</span> <span class="variable">max</span> <span class="operator">=</span> <span class="number">1</span>;</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; N; i += <span class="number">1</span>) &#123;</span><br><span class="line">    dp[i] = <span class="number">1</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j &lt; i; j += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (nums[i] &gt; nums[j]) &#123;</span><br><span class="line">            dp[i] = Math.max(dp[i], dp[j] + <span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (dp[i] &gt; max) &#123;</span><br><span class="line">        max = dp[i];</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line">System.out.println(max);</span><br></pre></td></tr></table></figure><p>如何理解<code>dp[i] = Math.max(dp[i], dp[j] + 1)</code>?<br />这里dp[j]存储的是以j为结尾的LIS，而+1代表的是dp[i]自己。<br />我们通过计算出前面的所有dp[j]，最后只需要看对于每个nums[j]，它是否小于nums[i]，小于，就添加一个就可以了。</p><h1 id="p2782-友好城市"><a class="markdownIt-Anchor" href="#p2782-友好城市"></a> P2782 友好城市</h1><p>#贪心 #LIS最优解法<br />友好城市可以转换为一个LIS问题：将北岸城市按照坐标顺序排序后，求北岸城市对应南岸城市的坐标LIS（南岸城市坐标必须递增，不递增就是交叉造桥），这就是不交叉情况下能够建筑的最多桥梁数。<br />传统的DP写法需要<span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>O</mi><mo stretchy="false">(</mo><msup><mi>n</mi><mn>2</mn></msup><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">O(n^2)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.064108em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.02778em;">O</span><span class="mopen">(</span><span class="mord"><span class="mord mathdefault">n</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141079999999999em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span>的时间复杂度，会超时；下面介绍LIS的最优解法：贪心+二分。</p><h2 id="贪心法求解lis"><a class="markdownIt-Anchor" href="#贪心法求解lis"></a> 贪心法求解LIS</h2><p>对于一个序列sequence，遍历sequence[i]，维护一个上升序列数组，使其每个元素尽可能地小（这样整个序列就尽可能长），遍历结束，这个数组就是LIS。<br />具体的算法实现是：对于每个sequence[i]，查找它在贪心上升序列greedy中应该插入的位置（维持序列上升的位置），并替换原来的更大的元素，如果不存在更大的元素，在末尾追加该元素。最后，greedy就是LIS，greedy的长度就是能够建筑合法桥梁的最大值。</p><h3 id="优化dp思路交换状态与状态值"><a class="markdownIt-Anchor" href="#优化dp思路交换状态与状态值"></a> 优化DP思路：交换状态与状态值</h3><p>原来的DP是这样表示：<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>d</mi><mi>p</mi><mo stretchy="false">[</mo><mi>i</mi><mo stretchy="false">]</mo><mtext> 表示 末尾元素 为</mtext><mi>c</mi><mi>i</mi><mi>t</mi><mi>i</mi><mi>e</mi><mi>s</mi><mo stretchy="false">[</mo><mi>i</mi><mo stretchy="false">]</mo><mtext>的元素的</mtext><mi>L</mi><mi>I</mi><mi>S</mi><mtext> 长度</mtext></mrow><annotation encoding="application/x-tex">dp[i]\text{ 表示\ 末尾元素\ 为}cities[i]\text{的元素的}LIS\text{ 长度}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault">d</span><span class="mord mathdefault">p</span><span class="mopen">[</span><span class="mord mathdefault">i</span><span class="mclose">]</span><span class="mord text"><span class="mord"> </span><span class="mord cjk_fallback">表示</span><span class="mord"> </span><span class="mord cjk_fallback">末尾元素</span><span class="mord"> </span><span class="mord cjk_fallback">为</span></span><span class="mord mathdefault">c</span><span class="mord mathdefault">i</span><span class="mord mathdefault">t</span><span class="mord mathdefault">i</span><span class="mord mathdefault">e</span><span class="mord mathdefault">s</span><span class="mopen">[</span><span class="mord mathdefault">i</span><span class="mclose">]</span><span class="mord text"><span class="mord cjk_fallback">的元素的</span></span><span class="mord mathdefault">L</span><span class="mord mathdefault" style="margin-right:0.07847em;">I</span><span class="mord mathdefault" style="margin-right:0.05764em;">S</span><span class="mord text"><span class="mord"> </span><span class="mord cjk_fallback">长度</span></span></span></span></span><br />交换“末尾元素”与“长度”后：<br /><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>g</mi><mi>r</mi><mi>e</mi><mi>e</mi><mi>d</mi><mi>y</mi><mo stretchy="false">[</mo><mi>i</mi><mo stretchy="false">]</mo><mtext> 表示 长度 为</mtext><mi>i</mi><mo>+</mo><mn>1</mn><mtext>的</mtext><mi>I</mi><mi>S</mi><mtext>的 末尾元素 的最小值</mtext></mrow><annotation encoding="application/x-tex">greedy[i]\text{ 表示\ 长度\ 为}i+1\text{的}IS\text{的\ 末尾元素\ 的最小值}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathdefault" style="margin-right:0.03588em;">g</span><span class="mord mathdefault" style="margin-right:0.02778em;">r</span><span class="mord mathdefault">e</span><span class="mord mathdefault">e</span><span class="mord mathdefault">d</span><span class="mord mathdefault" style="margin-right:0.03588em;">y</span><span class="mopen">[</span><span class="mord mathdefault">i</span><span class="mclose">]</span><span class="mord text"><span class="mord"> </span><span class="mord cjk_fallback">表示</span><span class="mord"> </span><span class="mord cjk_fallback">长度</span><span class="mord"> </span><span class="mord cjk_fallback">为</span></span><span class="mord mathdefault">i</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord">1</span><span class="mord text"><span class="mord cjk_fallback">的</span></span><span class="mord mathdefault" style="margin-right:0.07847em;">I</span><span class="mord mathdefault" style="margin-right:0.05764em;">S</span><span class="mord text"><span class="mord cjk_fallback">的</span><span class="mord"> </span><span class="mord cjk_fallback">末尾元素</span><span class="mord"> </span><span class="mord cjk_fallback">的最小值</span></span></span></span></span></p><p>代码实现如下：</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    ... <span class="comment">// 处理输入，按北岸城市坐标cities[i].source排序</span></span><br><span class="line">    </span><br><span class="line">    <span class="type">int</span>[] greedy = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line">    <span class="type">int</span> <span class="variable">len</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; N; i += <span class="number">1</span>) &#123; <span class="comment">// 顺序遍历排好序的北岸城市坐标</span></span><br><span class="line">        <span class="type">int</span> <span class="variable">target</span> <span class="operator">=</span> cities[i].target; <span class="comment">// 北岸城市对应的友好城市坐标</span></span><br><span class="line">        <span class="type">int</span> <span class="variable">index</span> <span class="operator">=</span> lowerBound(greedy, len-<span class="number">1</span>, target); <span class="comment">// 注意要传入len-1</span></span><br><span class="line">        <span class="keyword">if</span> (index == len) &#123; <span class="comment">// 追加元素</span></span><br><span class="line">            greedy[len++] = target;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123; <span class="comment">// 找到递增序列位置，替换</span></span><br><span class="line">            greedy[index] = target;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    System.out.println(len);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 寻找target应该插入到递增序列nums的下标位置</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">lowerBound</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span> end, <span class="type">int</span> target)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">start</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">while</span> (start &lt;= end) &#123;</span><br><span class="line">        <span class="type">int</span> <span class="variable">mid</span> <span class="operator">=</span> start + (end - start) / <span class="number">2</span>;</span><br><span class="line">        <span class="keyword">if</span> (nums[mid] &gt;= target) &#123;</span><br><span class="line">            end = mid - <span class="number">1</span>;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            start = mid + <span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> start;</span><br><span class="line">&#125;</span><br><span class="line"></span><br></pre></td></tr></table></figure><h1 id="p1091-noip-2004-提高组-合唱队形"><a class="markdownIt-Anchor" href="#p1091-noip-2004-提高组-合唱队形"></a> P1091 [NOIP 2004 提高组] 合唱队形</h1><p>#双向LIS<br />合唱队形可以看成求两边LIS之和的最大值。此时总人数减去LIS之和的最大值，就是最少出列队员数。<br />这里要注意当前index+1的值才是正确的长度。len标记的是数组的总长度，但是index会动态更新寻找更小值并做替换。当index找到最小值时，后面的更大值是在index以前的，不属于当前下标i+1结尾的IS长度。</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="type">int</span>[] gdUp = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line"><span class="type">int</span> <span class="variable">lenUp</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"><span class="type">int</span>[] lenUps = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; N; i += <span class="number">1</span>) &#123;</span><br><span class="line">  <span class="type">int</span> <span class="variable">index</span> <span class="operator">=</span> lowerBound(gdUp, lenUp-<span class="number">1</span>, members[i]);</span><br><span class="line">  <span class="keyword">if</span> (index == lenUp) &#123;</span><br><span class="line">    gdUp[lenUp++] = members[i];</span><br><span class="line">  &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">    gdUp[index] = members[i];</span><br><span class="line">  &#125;</span><br><span class="line">  lenUps[i] = index + <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="type">int</span>[] gdDown = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line"><span class="type">int</span> <span class="variable">lenDown</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"><span class="type">int</span>[] lenDowns = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> N-<span class="number">1</span>; i &gt;= <span class="number">0</span>; i -= <span class="number">1</span>) &#123;</span><br><span class="line">  <span class="type">int</span> <span class="variable">index</span> <span class="operator">=</span> lowerBound(gdDown, lenDown-<span class="number">1</span>, members[i]);</span><br><span class="line">  <span class="keyword">if</span> (index == lenDown) &#123;</span><br><span class="line">    gdDown[lenDown++] = members[i];</span><br><span class="line">  &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">    gdDown[index] = members[i];</span><br><span class="line">  &#125;</span><br><span class="line">  lenDowns[i] = index + <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="type">int</span> <span class="variable">max</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line"><span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; N; i += <span class="number">1</span>) &#123;</span><br><span class="line">  <span class="keyword">if</span> (lenUps[i] + lenDowns[i] - <span class="number">1</span> &gt; max) &#123;</span><br><span class="line">    max = lenUps[i] + lenDowns[i] - <span class="number">1</span>;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">System.out.println(N - max);</span><br></pre></td></tr></table></figure><h1 id="p1020-noip-1999-提高组-导弹拦截"><a class="markdownIt-Anchor" href="#p1020-noip-1999-提高组-导弹拦截"></a> P1020 [NOIP 1999 提高组] 导弹拦截</h1><p>#最长不递增子序列<br />做这题各种WA让我非常confusing，仔细研究后发现是我没有理解导弹拦截的规则（可以拦截相等高度！），真所谓“失之毫厘，谬以千里”。原理其实很简单：</p><ol><li>导弹系统可以拦截的最多导弹数，是一个<strong>最长不严格递减子序列</strong>（导弹高度不需要严格递减、可以相等），在题目要求的数据规模下，必须使用贪心+二分解法，转化为逆序求<strong>最长不严格递增子序列</strong>。</li><li>最少需要多少导弹拦截系统？一个系统只能拦截比前一个导弹高度更低的导弹，那么每出现一个比之前所有高度都更高的导弹，之前的系统都不能拦截。这就是LIS！</li></ol><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    ... <span class="comment">// 处理输入</span></span><br><span class="line">    </span><br><span class="line">    <span class="type">int</span>[] gdDown = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line">    <span class="type">int</span> <span class="variable">maxMissile</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> N-<span class="number">1</span>; i &gt;= <span class="number">0</span>; i -= <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="type">int</span> <span class="variable">index</span> <span class="operator">=</span> upperBound(gdDown, maxMissile - <span class="number">1</span>, nums[i]);</span><br><span class="line">        <span class="keyword">if</span> (index == maxMissile) &#123;</span><br><span class="line">            gdDown[maxMissile++] = nums[i];</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            gdDown[index] = nums[i];</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    <span class="type">int</span>[] gdUp = <span class="keyword">new</span> <span class="title class_">int</span>[N];</span><br><span class="line">    <span class="type">int</span> <span class="variable">numSystems</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; N; i += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="type">int</span> <span class="variable">index</span> <span class="operator">=</span> lowerBound(gdUp, numSystems - <span class="number">1</span>, nums[i]);</span><br><span class="line">        <span class="keyword">if</span> (index == numSystems) &#123;</span><br><span class="line">            gdUp[numSystems++] = nums[i];</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            gdUp[index] = nums[i];</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    </span><br><span class="line">    System.out.println(maxMissile + <span class="string">&quot;\n&quot;</span> + numSystems);       </span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 找到第一个大于该数（不管有没有找到，允许gd里的数重复）的位置</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">upperBound</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span> end, <span class="type">int</span> target)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">start</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">while</span> (start &lt;= end) &#123;</span><br><span class="line">        <span class="type">int</span> <span class="variable">mid</span> <span class="operator">=</span> start + (end - start) / <span class="number">2</span>;</span><br><span class="line">        <span class="keyword">if</span> (nums[mid] &lt;= target) &#123; </span><br><span class="line">            start = mid + <span class="number">1</span>;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            end = mid - <span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> start;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 找到第一个大于（没找到，在这个位置插入）/等于（找到，在这个位置替换）该数的位置</span></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="type">int</span> <span class="title function_">lowerBound</span><span class="params">(<span class="type">int</span>[] nums, <span class="type">int</span> end, <span class="type">int</span> target)</span> &#123;</span><br><span class="line">    <span class="type">int</span> <span class="variable">start</span> <span class="operator">=</span> <span class="number">0</span>;</span><br><span class="line">    <span class="keyword">while</span> (start &lt;= end) &#123;</span><br><span class="line">        <span class="type">int</span> <span class="variable">mid</span> <span class="operator">=</span> start + (end - start) / <span class="number">2</span>;</span><br><span class="line">        <span class="keyword">if</span> (nums[mid] &gt;= target) &#123;</span><br><span class="line">            end = mid - <span class="number">1</span>;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            start = mid + <span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> start;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h1 id="p1086-noip-2004-普及组-花生采摘"><a class="markdownIt-Anchor" href="#p1086-noip-2004-普及组-花生采摘"></a> P1086 [NOIP 2004 普及组] 花生采摘</h1><p>#模拟<br />这题是简单的模拟题，按照题目要求完成即可。不过有一些小细节需要注意：</p><ul><li>数组下标从0开始，但是坐标不能为0，否则会计算错误。<br />代码使用了TreeMap来自动排序所有的花生植株，所以看起来不太直观。</li></ul><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    ... <span class="comment">// 处理输入</span></span><br><span class="line">    </span><br><span class="line">    findPeanut(map, M, N);</span><br><span class="line">    <span class="comment">// TreeMap升序排序，取当前Max要从最后取</span></span><br><span class="line">    Map.Entry&lt;Integer, Point&gt; pre = peanuts.pollLastEntry(); </span><br><span class="line"></span><br><span class="line">    <span class="keyword">if</span> (<span class="number">2</span> * pre.getValue().x + <span class="number">1</span> &gt; K) &#123; <span class="comment">// 往返路程 + 采摘的总消耗</span></span><br><span class="line">        System.out.println(<span class="number">0</span>);</span><br><span class="line">        <span class="keyword">return</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    K -= pre.getValue().x + <span class="number">1</span>; <span class="comment">// 进入花生田 + 摘花生，不返回的总消耗</span></span><br><span class="line">    <span class="type">int</span> <span class="variable">cnt</span> <span class="operator">=</span> pre.getKey();</span><br><span class="line">    peanuts.remove(pre.getKey());</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>) &#123;</span><br><span class="line">        Map.Entry&lt;Integer, Point&gt; cur = peanuts.pollLastEntry();</span><br><span class="line">        <span class="keyword">if</span> (cur == <span class="literal">null</span>) <span class="keyword">break</span>;</span><br><span class="line">        <span class="type">int</span> <span class="variable">cost</span> <span class="operator">=</span> Math.abs(pre.getValue().x - cur.getValue().x) +</span><br><span class="line">                   Math.abs(pre.getValue().y - cur.getValue().y) + <span class="number">1</span>;</span><br><span class="line">        <span class="keyword">if</span> (cost + cur.getValue().x &gt; K) &#123; <span class="comment">// 摘花生 + 返回路边的总消耗</span></span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        K -= cost;</span><br><span class="line">        cnt += cur.getKey();</span><br><span class="line">        peanuts.remove(pre.getKey());</span><br><span class="line">        pre = cur;</span><br><span class="line">    &#125;</span><br><span class="line">    System.out.println(cnt);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 记录每株花生的数量和坐标</span></span><br><span class="line"><span class="keyword">static</span> TreeMap&lt;Integer, Point&gt; peanuts = <span class="keyword">new</span> <span class="title class_">TreeMap</span>&lt;&gt;();</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title function_">findPeanut</span><span class="params">(<span class="type">int</span>[][] map, <span class="type">int</span> M, <span class="type">int</span> N)</span> &#123;</span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">i</span> <span class="operator">=</span> <span class="number">0</span>; i &lt; M; i += <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">for</span> (<span class="type">int</span> <span class="variable">j</span> <span class="operator">=</span> <span class="number">0</span>; j &lt; N; j += <span class="number">1</span>) &#123;</span><br><span class="line">            <span class="keyword">if</span> (map[i][j] != <span class="number">0</span>) &#123;</span><br><span class="line">                peanuts.put(map[i][j], <span class="keyword">new</span> <span class="title class_">Point</span>(i + <span class="number">1</span>, j + <span class="number">1</span>)); <span class="comment">// 注意，坐标从1开始</span></span><br><span class="line">                map[i][j] = <span class="number">0</span>;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;   </span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>]]></content>
    
    
    <summary type="html">&lt;h1 id=&quot;p1004-noip-2000-提高组-方格取数&quot;&gt;&lt;a class=&quot;markdownIt-Anchor&quot; href=&quot;#p1004-noip-2000-提高组-方格取数&quot;&gt;&lt;/a&gt; P1004 [NOIP 2000 提高组] 方格取数&lt;/h1&gt;
&lt;p&gt;#走两次dp&lt;br /&gt;
如果只走一次，这题是非常经典的DP。但是要走两次，就变得非常有难度。&lt;br /&gt;
首先，可以简单地推广：要走两次，dp就存四个下标：&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt;[][][][] dp = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;title class_&quot;&gt;int&lt;/span&gt;[N][N][N][N];&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;我们只需要遍历所有可能，并且比较四种走法（同下、同右、一下一右），取最大值就可以了。&lt;br /&gt;
注意，一个数只能取一次，需要一个判断防止重复取数。&lt;/p&gt;
&lt;figure class=&quot;highlight java&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;i1&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; i1 &amp;lt; N; i1 += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;i2&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; i2 &amp;lt; N; i2 += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;j1&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; j1 &amp;lt; N; j1 += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;for&lt;/span&gt; (&lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;j2&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;; j2 &amp;lt; N; j2 += &lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;type&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;variable&quot;&gt;step&lt;/span&gt; &lt;span class=&quot;operator&quot;&gt;=&lt;/span&gt; map[i1][j1];&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; (i2 != i1 &amp;amp;&amp;amp; j2 != j1) step += map[i2][j2];&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                dp[i1][j1][i2][j2] = &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    Math.max(dp[i1-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][j1][i2-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][j2], &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    Math.max(dp[i1-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][j1][i2][j2-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;], &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                    Math.max(dp[i1][j1-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][i2-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][j2], &lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                             dp[i1][j1-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][i2][j2-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;]))) + step;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;System.out.println(dp[N-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][N-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][N-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;][N-&lt;span class=&quot;number&quot;&gt;1&lt;/span&gt;]);&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;当然，4个循环时间复杂度太高了。我们可以用一个&lt;code&gt;k == i1 + j1 == i2 + j2&lt;/code&gt;来减少一重循环。&lt;br /&gt;
这个k利用得很巧妙，因为每次要么向下走，要么向右走，所以&lt;code&gt;k-1 == i-1 + j == i + j-1&lt;/code&gt;，全程使用&lt;code&gt;k-1&lt;/code&gt;就能代表所有情况。&lt;/p&gt;</summary>
    
    
    
    
    <category term="笔记" scheme="http://simuleite.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
    
  </entry>
  
</feed>