Delete all lines not containing string in Sublime

L

2

26

I recently got a bruteforce on my website, and wanted to write it down somewhere. The bad new is that the log file itself are 1,4 GB large (4338995 Lines) and I haven't got the logrotate fully working yet.

So I was wondering how I could remove all lines that does not contain a certain string in Sublime. Since the file is so big, it's nearly unreadable and I can't really get the whole view of it. It contains both Normal users, and two bruteforces from two different IP addresses (Probably same person).

It looks something like this (All personal info and IP addresses has been changed.) :

163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /box1_rhs/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /isaac_working/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
66.29.166.6 - - [28/Apr/2017:13:00:06 +0200] "GET /index.php HTTP/1.1" 200 2898 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /title_bykergrove_red/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /games_title/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
66.29.166.6 - - [28/Apr/2017:13:00:06 +0200] "GET /info.php HTTP/1.1" 200 565 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /box1_btm/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /coast_gal_bamburgh-thumb/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /games_pic2/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /coast_gal_tentsmuir-thumb/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /pannel_bot/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /but_go_red/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /coast_gal_badbea-thumb/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /top_girl/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
61.68.207.144 - - [28/Apr/2017:13:00:06 +0200] "GET /s/ HTTP/1.1" 200 9707 "http://google.com/search?q=s06e13" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /pannel_poles_bottom/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /box2_rhs/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /watch_animals/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /pets_pic4/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /boy/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /box2_top/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /pets_title/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /coast_gal_whitby-thumb/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:06 +0200] "HEAD /box2_schoolsout_paramedic/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /rws_sign/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
12.180.245.229 - - [28/Apr/2017:13:00:07 +0200] "GET /browse.php HTTP/1.1" 200 3819 "https://www.google.com.au/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /box2_btm/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /serious_amazon/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /box3_noproblem_textbullying/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /ramblings12_home/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"
163.33.74.115 - - [28/Apr/2017:13:00:07 +0200] "HEAD /chain_cat/ HTTP/1.1" 404 157 "-" "DirBuster-0.12 (http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project)"

As you can see, I would like to remove all lines not containing: "163.33.74.115" because I want both his bruteforce, and his casually browsing. I tried Pressing CTRL+H (Search & Replace) and used this code:

^((?!163\.33\.74\.115).)*$

But the program didn't do anything even tho I saw everything else than the IP was highlighted.

How do I do this?

Luganda answered 30/4, 2017 at 23:59 Comment(0)

L

39

For Sublime use:

1 - CTRL+H
2 - Click Regular Expressions (check ps below)
3 - Find What: ^163.33.74.115.*\n or ^(?!163.33.74.115).*\n for inverted matches
4 - Replace With: blank
5 - Click Replace All

GREP ANSWER:

The answer above should work fine, but I'd rather use grep, which is bundled with linux and mac, for windows get it here, i.e.:

1 - All lines except the ones containing 163.33.74.115:

grep -v 163.33.74.115 original.log > attack.log

2 - All lines containing 163.33.74.115:

grep 163.33.74.115 original.log > attack.log

Options:

-v, --invert-match        select non-matching lines

Longlegged answered 1/5, 2017 at 1:52 Comment(12)

Is this related to Linux, sublime or both? – Luganda 1/5, 2017 at 2:4

You can use it natively on any linux or mac – Longlegged 1/5, 2017 at 2:5

Ok, I'll transfer the file back to the server and do the command – Luganda 1/5, 2017 at 2:5

Are you on windows ? – Longlegged 1/5, 2017 at 2:5

Yes. But server is running linux. I used a editing program called sublime, and was wondering how you would do it with that program. – Luganda 1/5, 2017 at 2:6

I've updated the answer, please let me know if it worked – Longlegged 1/5, 2017 at 2:15

Hmm, It needs to be inverted – Luganda 1/5, 2017 at 2:21

for the opposite use: ^(?!163.33.74.115).*\n – Longlegged 1/5, 2017 at 2:23

The linux command worked, and I'll check your answer once I get sublime confirmed. – Luganda 1/5, 2017 at 2:29

Working with files as big as 1 GB is problematic. So I can't fully confirm that it works because it keeps crashing. But I can confirm it working on the small scale. Beautiful answer! – Luganda 1/5, 2017 at 2:40

I feel like there's got to be an easier way to do this with sublime using their multiple line edit (multi-edit) capability with multiple cursors – Euphrosyne 14/9, 2017 at 21:2

@Euphrosyne - grep is the easiest and best performant solution. – Longlegged 1/6, 2020 at 4:43

C

29

A quicker option is to just use the 'Find All' option: It selects all the matches for you, so you can copy them.

Ctrl+F
Match the lines you want using simple, positive-logic regex: .*163.33.74.115.*
Click "Find All"
Ctrl+C > Open New document > Ctrl+V

The advantage here is that you don't have to remember the regex syntax for negative lookahead -- which is even trickier if you're trying to match on something not at the beginning of the line.

Companionship answered 26/2, 2019 at 16:3 Comment(0)

Recommended topics

Hot tags