Difference between revisions of "Merge multiple text files into one file"

From LemonWiki共筆
Jump to navigation Jump to search
Tags: Mobile edit Mobile web edit
Line 18: Line 18:
== References ==
== References ==
<reference />
<references />
[[Category:Data Science]]
[[Category:Data Science]]
[[Category:Text file processing]]
[[Category:Text file processing]]

Revision as of 06:04, 6 March 2020


Step 1: check the last line of text file is newline[1]

Step 2: Merge the content

  • copy *.txt > bundle.txt or copy file1.txt file2.txt > bundle.txt on Win Os windows.png
  • cat *.txt > bundle.txt or cat file1.txt file2.txt > bundle.txt on Linux Os linux.png & Mac Os mac.png [2][3]

Step 3: (optional) Remove the duplicated lines

  • sort -us -o bundle_unique.txt bundle.txt[4] OS: Linux Os linux.png , cygwin of Win Os windows.png "-u means Unique keys; -s means stable sort; -o means output" quoted from sort manual.

Step 4: (optional) Remove the heading of CSV file

Step 5: Verify the merge

  • count number of lines wc -l filename[5]