[SOLVED]help with sed command

xptandre7 · 2016-09-07 18:47:02

Hello,

I have a .txt that has multiple lines reading:

foo( unknown_string, unknown_string, unknown_string, unknown_string, unknown_string and so on (...)

I need to change the .txt contents to dispose of two strings that can be anything and keep all text following the second comma after "known_string("

Im using sed command, tried to search the web but still haven't cracked it. How do i specify sed to look only after known string and dischard everything up to the second comma?

Thanks,

---Mod edit: this is a help request not a Tip ---

Last edited by xptandre7 (2016-09-08 21:45:20)

damo · 2016-09-07 19:14:36

Could you post some example text, using [ code ] tags?

It may be that using awk or other tools is easier?

Also, to ensure that this isn't an "XY" problem, what are you actually trying to do?

Last edited by damo (2016-09-07 19:18:18)

xptandre7 · 2016-09-07 19:47:20

damo wrote:

Could you post some example text, using [ code ] tags?
It may be that using awk or other tools is easier?
Also, to ensure that this isn't an "XY" problem, what are you actually trying to do?

Im trying to change a .sql file that contains multiple INSERT INTO lines with multiple columns. My .sql file is this:

INSERT INTO my_table( "232323", "324233", "333", "horayy");
INSERT INTO my_table( "4454", "556", "34", "any_name");
(...)

I need to change it to:

INSERT INTO my_table( "333", "horayy");
INSERT INTO my_table( "34", "any_name");
(...)

So i need to dischard the first two columns that can be anything (integers) and keep the rest of the line.

Thanks,

damo · 2016-09-07 21:28:43

This awk one-liner works, but is a bit ugly:

Input file, "sql.txt"

INSERT INTO my_table( "232323", "324233", "333", "horayy");
INSERT INTO my_table( "4454", "556", "34", "any_name");
INSERT INTO my_table( "232323", "324233", "333", "this");
INSERT INTO my_table( "4454", "556", "34", "that");

Awk one-liner

$ awk -F'[(,]' '{print $1"("$4","$NF}' sql.txt  > sql2.txt

Output file, "sql2.txt"

INSERT INTO my_table( "333" , "horayy");
INSERT INTO my_table( "34" , "any_name");
INSERT INTO my_table( "333" , "this");
INSERT INTO my_table( "34" , "that");

Last edited by damo (2016-09-07 21:33:08)

johnraff · 2016-09-08 03:10:32

Damo's awk command is neat and short, but if you want to do it with sed and a regular expression:

sed -r 's/(^INSERT INTO my_table\()([^,]*, ){2}(.*$)/\1 \3/' sql.txt

gives

INSERT INTO my_table( "333", "horayy");
INSERT INTO my_table( "34", "any_name");
INSERT INTO my_table( "333", "this");
INSERT INTO my_table( "34", "that");

This only operates on lines that match the whole expression. It outputs the first section of the string, then ignores two sections of '[^,]*, ' (ie any number of anything-but-a-comma + comma + space) then outputs a space and the rest of the string. I don't know if it's any less ugly.

damo · 2016-09-08 03:38:19

johnraff wrote:

.... I don't know if it's any less ugly.

Even uglier, as only sed can manage!

I meant that the awk version uses "(" and "," as delimiters, but then has to replace the characters when printing the fields. It isn't very elegant

johnraff · 2016-09-08 07:03:31

In fact the two commands are doing different things, but both produce the output the OP wanted in this case.

(Sed's beauty is something few can appreciate...)

xptandre7 · 2016-09-08 12:51:48

Hi everyone,

Thank you all for the replies, i managed to get what i want.

Cheers,

Head_on_a_Stick · 2016-09-08 20:48:42

Which solution did you use in the end?

My curiosity is piqued

johnraff wrote:

Sed's beauty is something few can appreciate...

Yes indeed, `sed` is Turing complete [1] and if I ever achieve the silicon metempsychosis then that is the language in which I wish to be encoded 8)

http://www.catonmat.net/ftp/sed/turing.txt

xptandre7 · 2016-09-08 20:53:17

Head_on_a_Stick wrote:

Which solution did you use in the end?
My curiosity is piqued
johnraff wrote:
Sed's beauty is something few can appreciate...
Yes indeed, `sed` is Turing complete [1] and if I ever achieve the silicon metempsychosis then that is the language in which I wish to be encoded 8)
http://www.catonmat.net/ftp/sed/turing.txt

I used sed! Had to change it to fit other parts of the file (the "whole" line was different) but sed did the trick!

Head_on_a_Stick · 2016-09-08 21:07:08

OK, thanks for reporting back.

Please add [SOLVED] to the thread title if you consider it so.

Head_on_a_Stick · 2016-09-08 21:18:02

xptandre7 wrote:

Had to change it to fit other parts of the file (the "whole" line was different)

Can you share the exact command?

It may prove useful for others.

xptandre7 · 2016-09-08 21:41:47

Head_on_a_Stick wrote:

xptandre7 wrote:
Had to change it to fit other parts of the file (the "whole" line was different)
Can you share the exact command?
It may prove useful for others.

The command was the same, but the line

INSERT INTO my_table

Was changed to fit others lines of the file. The command was the same but as johnraff said:

johnraff wrote:

This only operates on lines that match the whole expression.

So all i had to do was change the line to fit any line followed by the string i wanted to consider:

sed -i -r 's/([^*]VALUES \()([^,]*, ){2}(.*$)/\1 \3/' my_file.sql

So anything before "VALUES" is kept.

#1 2016-09-07 18:47:02

[SOLVED]help with sed command

#2 2016-09-07 19:14:36

Re: [SOLVED]help with sed command

#3 2016-09-07 19:47:20

Re: [SOLVED]help with sed command

#4 2016-09-07 21:28:43

Re: [SOLVED]help with sed command

#5 2016-09-08 03:10:32

Re: [SOLVED]help with sed command

#6 2016-09-08 03:38:19

Re: [SOLVED]help with sed command

#7 2016-09-08 07:03:31

Re: [SOLVED]help with sed command

#8 2016-09-08 12:51:48

Re: [SOLVED]help with sed command

#9 2016-09-08 20:48:42

Re: [SOLVED]help with sed command

#10 2016-09-08 20:53:17

Re: [SOLVED]help with sed command

#11 2016-09-08 21:07:08

Re: [SOLVED]help with sed command

#12 2016-09-08 21:18:02

Re: [SOLVED]help with sed command

#13 2016-09-08 21:41:47

Re: [SOLVED]help with sed command

Board footer