Keyboard Ninja: Concatenate Multiple Text Files in Windows
You have a directory full of log files that you want to import into Excel or a database so you can do some processing on them… but there are hundreds of files… how do you make them into a single file?
Answer: Pull out your DOS hat, open a command prompt, and then use the "for" command.
The syntax works something like this:
for <variablename> in (<directorylisting>) do <command> <variablename>
So if you wanted to append all of the *.log files in a directory, you'd use the "type" command and then pipe it into a single file using the >> operator.
The difference between >> and > is that the former appends data to the end of the file, and the latter will completely replace the file, which would be pointless for what we want to do.
So here's the command you'd run, assuming you are in the directory containing the log files.
for %f in (*.log) do type "%f" >> aggregate.txt
And yes, I actually just used this command for a project at work, which is why I'm writing up this article. =)
Random thought: What on earth would a DOS hat look like?
I just tried this. After all the .log files in the directory are added to the aggregate file, the entire aggregate file is then added to itself (because it is now a new file in the directory). Is there a way to keep this from happening if you would want to keep the aggregate file in the same directory that you are working in?
Skeeter:
That did not happen for me…. but your best bet is to output to a file with a different extension, like aggregate.txt
That way it wouldn't be included in the list of files in the directory.
I updated the article to reflect that as well….
You can also just use the DOS copy command:
copy /a *.log aggregate.txt
The /a option ensures that any ^Z's at end-of-file in the source files will not be copied to the destination file.
Much more easier with the copy DOS command and the whole aggregate file is not added at the end (happened to me too with the for method).
Thanks for this. It was a lifesaver when I needed to pull together a whole bunch of sql scripts into one big script.
Life Saver!!!!
LIFESAVER almost …
The "for %f in (*.log) do type "%f" >> aggregate.txt" technique concatenated all files but then multiplied everything by 2. There was also some issue with the last line from Report A and the first line in Report B.
The second command, "copy /a *.log aggregate.txt" worked best.
THANKS!
Brilliant!!! Thankyou thankyou. I was going cross-eyed manually importing all these text files into access one at a time.
Thanks!!!! This is a complete lifesaver!! So simple and effective, I got just the right data out of bazillion log files I had to go through.
This helped me lots. Thank you! The one thing I would add is that you can also use this technique in a batch file if you just use %%f instead of %f
brillant! thank you so much!