How can I print only those lines that appear exactly once in a file? E.g., given this file: mountain forest mountain eagle The output would be this, because the line mountain appears twice: for...

I am writing a script to manipulate a text file. First thing I want to do is check if duplicate entries exist and if so, ask the user whether we wants to keep or remove them. I know how to displa...

I often use sort | uniq -c to make count statistics. Now, if I have two files with such count statistics, I would like to put them together and add the counts. (I know I could append the original f...

I have a list with population, year, and county and I need to cut the list, and then find the number of uniq counties. The list starts off like this: #Population, Year, County 3900, 1969, Beaver...

How to do natural sort on uniq -c output? When the counts are <10, the uniq -c | sort output looks fine: alvas@ubi:~/testdir$ echo -e "aaa\nbbb\naa\ncd\nada\naaa\nbbb\naa\nccd\naa" > test.t...

I have a file like this: 80.13.178.2 80.13.178.2 80.13.178.2 80.13.178.2 80.13.178.1 80.13.178.3 80.13.178.3 80.13.178.3 80.13.178.4 80.13.178.4 80.13.178.7 I need to display unique entries for ...

He all, I have a file having some columns. I would like to do sort for column 2 then apply uniq for column 1. I found this post talking about sort and uniq for the same column but my problem is a ...

Getting "illegal byte sequence" error while trying to extract non English characters from a large file in MacOS bash shell. This is the script that I am trying to use: sed 's/[][a-z,0-9,A-Z,!@#\$%...

1.gui Qxx 16 2.gu Qxy 23 3.guT QWS 18 4.gui Qxr 21 i want to sort a file depending a value in the 3rd column, so i use: sort -rnk3 myfile 2.gu Qxy 23 4.gui Qxr 21 3.guT QWS 18 1.gui Qxx 16 ...

I have following query: Article.joins(:themes => [:users]).where(["articles.user_id != ?", current_user.id]).order("Random()").limit(15).uniq and gives me the error PG::Error: ERROR: for SEL...

I am using the uniq function exported by the module, List::MoreUtils to find the uniq elements in an array. However, I want it to find the uniq elements in a case insensitive way. How can I do that...

What is the difference between the following two commands? sort -u FILE sort FILE | uniq

This must surely be a trivial task with awk or otherwise, but it's left me scratching my head this morning. I have a file with a format similar to this: pep> AEYTCVAETK 2 genes ADUm.1024,ADUm.5...

I'm sorry for the very noob question, but I'm kind of new to bash programming (started a few days ago). Basically what I want to do is keep one file with all the word occurrences of another file I...

I am currently having a very simple problem with capturing the output from a backticked shell command. I apologize that the problem is rather simple. I have some sorted array (@valid_runs) which ...

I'm running a Hadoop job using Hive actually that is supposed to uniq lines in many text files. In the reduce step, it chooses the most recently timestamped record for each key. Does Hadoop guaran...

Is there a difference in the order of uniq and sort when calling them in a shell script? I’m talking here about time- and space-wise. grep 'somePattern' | uniq | sort vs. grep 'somePattern' | s...