Skip to content

Commit d149f73

Browse files
myoshi2891myoshi2891
authored
Merge pull request #240 from myoshi2891/dev-from-macmini
feat: Add comprehensive Bash 10th line extraction tutorial with visua…
2 parents 4e0a35d + cb863a9 commit d149f73

5 files changed

Lines changed: 1225 additions & 557 deletions

File tree

‎Shell/Bash/Leetcode/192. Word Frequency/WordFrequency.ipynb‎

Lines changed: 363 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,363 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "43f342e3",
6+
"metadata": {},
7+
"source": [
8+
"# 192. Word Frequency - Bash解法\n",
9+
"\n",
10+
"## 問題概要\n",
11+
"\n",
12+
"テキストファイル `words.txt` から各単語の出現頻度を集計し、頻度の降順で出力する問題です。\n",
13+
"\n",
14+
"---\n",
15+
"\n",
16+
"## 解答（スクリプト版）\n",
17+
"\n",
18+
"`wordfreq.sh`（Bash, POSIX ツールのみ）\n",
19+
"\n",
20+
"```bash\n",
21+
"#!/usr/bin/env bash\n",
22+
"set -euo pipefail\n",
23+
"\n",
24+
"# 使い方: ./wordfreq.sh [path/to/words.txt]\n",
25+
"# 引数が未指定なら ./words.txt を読む\n",
26+
"input=\"${1:-words.txt}\"\n",
27+
"\n",
28+
"# 1) 全ての空白(スペース/タブ/改行など)を改行にし、連続空白は1つに圧縮\n",
29+
"# 2) ソート\n",
30+
"# 3) uniq -c で頻度集計\n",
31+
"# 4) 頻度(第1列)で数値降順ソート\n",
32+
"# 5) \"単語 頻度\" の並びに整形\n",
33+
"LC_ALL=C tr -s '[:space:]' '\\n' < \"$input\" \\\n",
34+
" | sort \\\n",
35+
" | uniq -c \\\n",
36+
" | sort -nr \\\n",
37+
" | awk '{print $2, $1}'\n",
38+
"```\n",
39+
"\n",
40+
"### 実行方法\n",
41+
"\n",
42+
"```bash\n",
43+
"chmod +x wordfreq.sh\n",
44+
"./wordfreq.sh # カレントの words.txt を集計\n",
45+
"# もしくは\n",
46+
"./wordfreq.sh /path/to/words.txt\n",
47+
"```\n",
48+
"\n",
49+
"---\n",
50+
"\n",
51+
"## 解答（パイプのみの1行版）\n",
52+
"\n",
53+
"```bash\n",
54+
"LC_ALL=C tr -s '[:space:]' '\\n' < words.txt | sort | uniq -c | sort -nr | awk '{print $2, $1}'\n",
55+
"```\n",
56+
"\n",
57+
"---\n",
58+
"\n",
59+
"## 入出力例\n",
60+
"\n",
61+
"### 入力 (`words.txt`)\n",
62+
"\n",
63+
"```text\n",
64+
"the day is sunny the the\n",
65+
"the sunny is is\n",
66+
"```\n",
67+
"\n",
68+
"### 出力\n",
69+
"\n",
70+
"```text\n",
71+
"the 4\n",
72+
"is 3\n",
73+
"sunny 2\n",
74+
"day 1\n",
75+
"```\n",
76+
"\n",
77+
"---\n",
78+
"\n",
79+
"## 処理フロー図解\n",
80+
"\n",
81+
"```mermaid\n",
82+
"flowchart LR\n",
83+
" A[\"words.txt<br/>入力ファイル\"] --> B[\"<code>tr -s &#91;:space:&#93; \\\\n</code><br/>全ての空白→改行<br/>連続空白を1つに圧縮\"]\n",
84+
" B --> C[\"<code>sort</code><br/>辞書順整列\"]\n",
85+
" C --> D[\"<code>uniq -c</code><br/>連続同一語をカウント\"]\n",
86+
" D --> E[\"<code>sort -nr</code><br/>頻度で降順ソート\"]\n",
87+
" E --> F[\"<code>awk &#123;print $2, $1&#125;</code><br/>「単語 頻度」形式に整形\"]\n",
88+
" F --> G[\"結果出力\"]\n",
89+
"```\n",
90+
"\n",
91+
"---\n",
92+
"\n",
93+
"## ステップ別の処理詳細\n",
94+
"\n",
95+
"### 入力データ\n",
96+
"\n",
97+
"```text\n",
98+
"the day is sunny the the\n",
99+
"the sunny is is\n",
100+
"```\n",
101+
"\n",
102+
"### ステップ1: `tr -s '[:space:]' '\\n'`\n",
103+
"\n",
104+
"全ての空白文字（スペース・タブ・改行）を改行に変換し、連続する空白は1つに圧縮します。\n",
105+
"\n",
106+
"```text\n",
107+
"the\n",
108+
"day\n",
109+
"is\n",
110+
"sunny\n",
111+
"the\n",
112+
"the\n",
113+
"the\n",
114+
"sunny\n",
115+
"is\n",
116+
"is\n",
117+
"```\n",
118+
"\n",
119+
"### ステップ2: `sort`\n",
120+
"\n",
121+
"単語を辞書順に整列します（`uniq -c` は連続した同一行のみカウントするため必須）。\n",
122+
"\n",
123+
"```text\n",
124+
"day\n",
125+
"is\n",
126+
"is\n",
127+
"is\n",
128+
"sunny\n",
129+
"sunny\n",
130+
"the\n",
131+
"the\n",
132+
"the\n",
133+
"the\n",
134+
"```\n",
135+
"\n",
136+
"### ステップ3: `uniq -c`\n",
137+
"\n",
138+
"連続する同一単語をカウントします。\n",
139+
"\n",
140+
"```text\n",
141+
" 1 day\n",
142+
" 3 is\n",
143+
" 2 sunny\n",
144+
" 4 the\n",
145+
"```\n",
146+
"\n",
147+
"### ステップ4: `sort -nr`\n",
148+
"\n",
149+
"頻度（第1列）で数値降順ソートします。\n",
150+
"\n",
151+
"```text\n",
152+
" 4 the\n",
153+
" 3 is\n",
154+
" 2 sunny\n",
155+
" 1 day\n",
156+
"```\n",
157+
"\n",
158+
"### ステップ5: `awk '{print $2, $1}'`\n",
159+
"\n",
160+
"「単語 頻度」の形式に整形します。\n",
161+
"\n",
162+
"```text\n",
163+
"the 4\n",
164+
"is 3\n",
165+
"sunny 2\n",
166+
"day 1\n",
167+
"```\n",
168+
"\n",
169+
"---\n",
170+
"\n",
171+
"## アルゴリズムの解説\n",
172+
"\n",
173+
"### なぜこの順番なのか？\n",
174+
"\n",
175+
"1. **`tr` で正規化**\n",
176+
" - 様々な空白文字（スペース・タブ・改行）を統一的に処理\n",
177+
" - 連続空白の圧縮により空行を防止\n",
178+
"\n",
179+
"2. **最初の `sort` が必須**\n",
180+
" - `uniq -c` は**連続した**同一行のみカウント\n",
181+
" - 事前に整列することで同じ単語を隣接させる\n",
182+
"\n",
183+
"3. **`uniq -c` で集計**\n",
184+
" - 連続する同一単語の出現回数をカウント\n",
185+
" - 出力形式: `<頻度> <単語>`\n",
186+
"\n",
187+
"4. **`sort -nr` で降順**\n",
188+
" - `-n`: 数値としてソート\n",
189+
" - `-r`: 降順（reverse）\n",
190+
"\n",
191+
"5. **`awk` で整形**\n",
192+
" - 列の順序を入れ替え: `$2 $1` → `<単語> <頻度>`\n",
193+
"\n",
194+
"---\n",
195+
"\n",
196+
"## 代替解法（awk メイン）\n",
197+
"\n",
198+
"`awk` の連想配列を使った方法：\n",
199+
"\n",
200+
"```bash\n",
201+
"awk '{for(i=1;i<=NF;i++) c[$i]++} END{for(w in c) print w, c[w]}' words.txt \\\n",
202+
" | LC_ALL=C sort -k2,2nr\n",
203+
"```\n",
204+
"\n",
205+
"### 処理の流れ\n",
206+
"\n",
207+
"1. `awk` で各単語をカウント\n",
208+
" - `NF`: 行内のフィールド数（空白区切り）\n",
209+
" - `c[$i]++`: 連想配列でカウント\n",
210+
"\n",
211+
"2. `END` ブロックで出力\n",
212+
" - `for(w in c)`: 全ての単語をループ\n",
213+
" - `print w, c[w]`: 単語と頻度を出力\n",
214+
"\n",
215+
"3. `sort -k2,2nr` で頻度降順ソート\n",
216+
" - `-k2,2`: 第2列（頻度）でソート\n",
217+
" - `n`: 数値ソート\n",
218+
" - `r`: 降順\n",
219+
"\n",
220+
"---\n",
221+
"\n",
222+
"## パフォーマンス最適化のポイント\n",
223+
"\n",
224+
"### 1. ロケール設定\n",
225+
"\n",
226+
"```bash\n",
227+
"LC_ALL=C\n",
228+
"```\n",
229+
"\n",
230+
"- C ロケールを使用することで `sort` が高速化\n",
231+
"- バイト単位の比較により安定した動作\n",
232+
"\n",
233+
"### 2. 空行の除去（必要に応じて）\n",
234+
"\n",
235+
"`tr -s` を使っていれば基本的に不要ですが、念のため：\n",
236+
"\n",
237+
"```bash\n",
238+
"... | grep -v '^$' | ...\n",
239+
"```\n",
240+
"\n",
241+
"### 3. 入力ファイルの柔軟な指定\n",
242+
"\n",
243+
"スクリプト版では引数でファイルパスを指定可能：\n",
244+
"\n",
245+
"```bash\n",
246+
"input=\"${1:-words.txt}\"\n",
247+
"```\n",
248+
"\n",
249+
"---\n",
250+
"\n",
251+
"## 応用例\n",
252+
"\n",
253+
"### 圧縮ファイルの処理\n",
254+
"\n",
255+
"```bash\n",
256+
"zcat compressed.txt.gz | tr -s '[:space:]' '\\n' | sort | uniq -c | sort -nr | awk '{print $2, $1}'\n",
257+
"```\n",
258+
"\n",
259+
"### ストリーム処理\n",
260+
"\n",
261+
"```bash\n",
262+
"curl -s https://example.com/text.txt | tr -s '[:space:]' '\\n' | sort | uniq -c | sort -nr | awk '{print $2, $1}'\n",
263+
"```\n",
264+
"\n",
265+
"### 大文字小文字を区別しない\n",
266+
"\n",
267+
"```bash\n",
268+
"LC_ALL=C tr -s '[:space:]' '\\n' < words.txt \\\n",
269+
" | tr '[:upper:]' '[:lower:]' \\\n",
270+
" | sort \\\n",
271+
" | uniq -c \\\n",
272+
" | sort -nr \\\n",
273+
" | awk '{print $2, $1}'\n",
274+
"```\n",
275+
"\n",
276+
"---\n",
277+
"\n",
278+
"## よくある質問\n",
279+
"\n",
280+
"### Q1: `tr -s` の `-s` オプションは何をする？\n",
281+
"\n",
282+
"**A:** `-s` (squeeze) は連続する文字を1つに圧縮します。\n",
283+
"\n",
284+
"```bash\n",
285+
"# 例: 連続するスペースを1つに\n",
286+
"echo \"a b c\" | tr -s ' '\n",
287+
"# 出力: a b c\n",
288+
"```\n",
289+
"\n",
290+
"### Q2: なぜ `LC_ALL=C` を使うのか？\n",
291+
"\n",
292+
"**A:** \n",
293+
"- ロケール依存の文字比較を避ける\n",
294+
"- バイト単位の比較で高速化\n",
295+
"- 環境による動作の違いを防ぐ\n",
296+
"\n",
297+
"### Q3: `uniq -c` の出力形式は？\n",
298+
"\n",
299+
"**A:** `<頻度><スペース><単語>` の形式で出力されます。\n",
300+
"\n",
301+
"```text\n",
302+
" 4 the\n",
303+
" 3 is\n",
304+
"```\n",
305+
"\n",
306+
"先頭にスペースが入るため、`awk` で列を入れ替える際は `$1` が頻度、`$2` が単語になります。\n",
307+
"\n",
308+
"---\n",
309+
"\n",
310+
"## Mermaid図の注意点\n",
311+
"\n",
312+
"Mermaid でコマンドを含むラベルを書く際の安全な記法：\n",
313+
"\n",
314+
"### 特殊文字のエスケープ\n",
315+
"\n",
316+
"- 角かっこ `[` `]` → `&#91;` `&#93;`\n",
317+
"- 波かっこ `{` `}` → `&#123;` `&#125;`\n",
318+
"- バックスラッシュ `\\` → `\\\\`\n",
319+
"- シングルクォート `'` → `&apos;`（必要な場合）\n",
320+
"\n",
321+
"### 推奨記法\n",
322+
"\n",
323+
"```mermaid\n",
324+
"flowchart LR\n",
325+
" A[\"ノード名\"] --> B[\"<code>コマンド</code><br/>説明文\"]\n",
326+
"```\n",
327+
"\n",
328+
"- ラベル全体を二重引用符 `[\"...\"]` で囲む\n",
329+
"- コマンド部分は `<code>` タグで囲む\n",
330+
"- 改行は `<br/>` を使用\n",
331+
"\n",
332+
"---\n",
333+
"\n",
334+
"## まとめ\n",
335+
"\n",
336+
"この問題の解法ポイント：\n",
337+
"\n",
338+
"1. **`tr`** で空白を正規化\n",
339+
"2. **`sort`** で同一単語を隣接させる\n",
340+
"3. **`uniq -c`** で頻度をカウント\n",
341+
"4. **`sort -nr`** で頻度降順ソート\n",
342+
"5. **`awk`** で出力形式を整形\n",
343+
"\n",
344+
"シンプルな POSIX ツールの組み合わせで効率的に処理できます。\n",
345+
"\n",
346+
"主な改善点：\n",
347+
"1. 重複セクションを完全に削除\n",
348+
"2. 構造を論理的に整理（問題→解答→詳細→応用）\n",
349+
"3. Mermaid図を1つに統一（安全な記法を使用）\n",
350+
"4. よくある質問セクションを追加\n",
351+
"5. 応用例を充実\n",
352+
"6. Mermaid記法の注意点を最後にまとめ"
353+
]
354+
}
355+
],
356+
"metadata": {
357+
"language_info": {
358+
"name": "python"
359+
}
360+
},
361+
"nbformat": 4,
362+
"nbformat_minor": 5
363+
}

0 commit comments

Comments
 (0)