Symmetric Matrix

I had a 4x4 symmetric matrix to be entered in Octave and I was lazy to feed redundant elements :), convincing myself that it is waste of time. Well, I spent more than that 'to be wasted' time browsing for how to enter symmetric matrices economically in Octave. Believe me, it was worth it. You never get bored fiddling with matrices and of course, Octave/Matlab.

What I found was not a single step solution, for that matter I could not get a single step way to do it at all. I was impressed by this solution for its elegance. Not just that, this solution caters to my requirement of feeding unique elements (either upper triangular or lower) exactly once. Okay, now on to the steps:

1) Say your matrix is as below:
    16    4    8    4
    4   10    8    4
    8    8   12   10
    4    4   10   12

Obviously symmetric - let's call this A


2) V=[16 4 10 8 8 12 4 4 10 12];

V holds all the upper (lower) triangular elements one column (row) at a time.

3) Create a triangular matrix of 1 of the same size as your A matrix as follows:

M = triu(ones(4))
M =

   1   1   1   1
   0   1   1   1
   0   0   1   1
   0   0   0   1

4) Replace the ones with elements from V as follows:

M(M==1)=V
M =

   16    4    8    4
    0   10    8    4
    0    0   12   10
    0    0    0   12


The M==1 test returns the indices of all places that has 1 in it, and the assignment takes care of replacing them with elements from V.

5) Add M with itself but after transposing and taking only the lower triangular portion (note M is an upper triangular one):

M = M + tril(M',-1)
M =

   16    4    8    4
    4   10    8    4
    8    8   12   10
    4    4   10   12

As you can see, M is A.

-1 in the tril command makes sure that you are leaving the main diagonal when retrieving the lower triangular portion of the transposed M.


So, as it looks it is a 4 step algorithm, but if you can create a script with these steps in it, it is only one step.

Now to credits: I got this tip from one of the threads of mathworks.com :), where else?!

Infinite Loop trace using GDB

Well, here is another use of GDB.
To locate the set of statements that get executed infinitely, as usual for the first few steps,

1) compile with -g option
2) pass the executable name to gdb command
This is take you to gdb environment.
3) Type 'run your-program-arguments'
Allow sufficient time to make your program get caught in that loop
4) Type ctrl+C
This will send SIGINT to your executable. You will see statements not making much sense.
5) Type 'backtrace'
6) Locate the frame number corresponding to your program or function name.
If the function is called, say, two levels deep from main, you will see all those called functions, but we are interested only in the function with least frame number. In otherwords, your main will have the highest frame number.
7) Type 'frame #(that number)'
This might not still show source lines from your program. Patience is required here.
8) Keep typing 'next' command or 'n' till you see your source lines.
9) Once you are see lines of your program, phew, there you go. Keep typing 'n' and you will see a bunch of lines getting executed again and again. You may want to check your local variables and other variable values for the expected values.

Thanks to unknownroad.com , I was able to fix this issue with less sweat.

Debug lessons

I was debugging using gdb and while the debug was going, I noted the changes to be made to the program in the program itself as comments and saved them - and, that was a MISTAKE. After such modifications, gdb internally sees the code which gave the executable but will show you only the new lines, which obviously we don't want. Phew!!

Load multiple files in matlab/octave

1) When you have multiples files of the same extension in the same directory, if you are looking for same operation to be done on them using a script, follow below for loading all of them at once to the Matlab/Octave environment:

files = dir('*.txt');
for i=1:length(files)
eval(['load ' files(i).name ' -ascii']);
end


I am getting a warning message - ignoring extra args - still trying to figure out what it means.
By the way, I got this tip from Mathworks itself, exactly here.

2) As a small example, say the operation you do is finding size of the files. The below snippet shows how to use eval command on set of files with common prefix but different numeral suffix that are loaded as above.

for ii=1994:1999
s=['size(tst' int2str(ii) ')'];
eval(s)
end


3) While searching for such related Matlab commands, I came across this interesting and useful not-to-dos in Matlab:here

Linux Util 2

1) You can try to place a tab between the quotes if you first press "<CTR> v" then the "<TAB>" key:

cut -f1 -d'<ctrl>v <tab>' filename

This way even <tab> character can also made as delimiter in shell scripting.

2) If you have a file where first column is list of years with author names against each year, to find the count of papers in each year:

cut -f1 -d' ' fname |uniq -c

3) To find all empty lines in the standard input:
grep ^$  or  grep -v .   

4) Hmm, long live blogging: I was looking for grep command to search for a text recursively from a given directory and this blogger helped:

http://www.geekality.net/2011/04/12/unix-recursive-search-for-text-in-files/

grep -rl "your_text"

5 ) To redirect the output of time (real, sys, usr) command onto redirect_file, instead of console.

(time your_command your_args) 2>> redirect_file

6) To search for a file's contents in another file:

grep -f searchfile tobesearchedfile -n --color=auto

Can remove the color flag. Got this color tip from Nixcraft

7) To list only the duplicate lines of a file:

uniq -d filename

8) To unzip a file that is only compressed with bz2 use

bunzip2 filename.bz2

9) To unzip things thar are compressed with .tar.bz2 use

tar -xvjpf filename.tar.bz2

I got the tips 8 and 9 from linuxquestions.org

10) Many a times, to know if the server I am working in is serving a lot of users, I type w in the shell prompt. This is useful if you are going to run a resource-greedy process. Now, if you are interested to write/talk to them, you may want to know their full name. The following command helps to get their full names from an administrative database:

getent passwd "jey"|cut -d':' -f5

If you are curious, an encouraging fact is man page of getent is small :) !!
Happy linuxing!!

11) Wow, after about 5 months (it's Jan 8, 2013 today), I am glad to be updating my blog. Yup, it also means I have failed to keep track of my new findings in web in this database. Anyways, the following tip was something very important to me because I realized I am spending too much time 'Alt+tab'ing to find the window I want to work in while working from home, logging to my office machine (essentially all are xterms).

To change the title of the xterm window to indicate something useful (in this example, your_string):
unset PROMPT_COMMAND
echo -ne "\033]0;your_string\007"

I had to call unset first because by default in my .bashrc file, PROMPT_COMMAND stores the username, machine and current working directory. It is important to observe the combination of characters around "your_string" and follow them strictly.

Awk and Sed

1) To search and replace in vi, you need not enter inside the file at all:
sed does the magic for you from command prompt itself. Tip here.

2) To alter the order of the columns in a file and write the output to another file:
awk '{print $3" "$1" "$2}' filename > newfilename

The above example takes a 3-columned file and changes the order as 1st col as 2nd, 2nd as 3rd and 3rd as 1st and writes the output to newfilename. There is no space between the double quotes and column indices. $n stands for specific column in the file in awk terminology. I am not sure if giving same source filename as the destination is going to work.


3) To check if the first column of a line in a file is some character and if yes, print the entire line:
awk '{if ($1~/e/) print $0}' filename

Here the first column has only one character throughout. $1 indicates the first column. In this specific example, if the first column contains a single character 'e', the command prints the entire line, indicated by $0. The single character can be replaced by a word too and there is no need for quotes around the word or character to be searched. There is no need for space around '~' character.

The default delimiter is space character. If you want to give a different delimiter, add -F\<delimter>. If the field you are comparing is a number, can give numeric relational operators in if condition like ==, !=, > etc.

4) Similar to case3 above, but to give more than one condition in the if  statement:
awk '{if (($1 ~ /e/) || ($1~/b/)) print $0}' filename

There are two important differences between case3 and case4. There are now enclosing '/' present around characters to be searched for and there is a logical OR sign '||'.

5) To search and replace character(s) from command prompt on a bunch
of files :

for((i=1;i<=30;i++)); do
sed -e 's/t:/ /g' filename$i > newfilename$i ; done

For 30 files that all have common prefix, search for character 't:' and replace them with a single space and write the output to a newfile with the same number

6)  Aug 13, 2013: To add a column of numbers generated as a result of say some intermediate linux util commands,

grep 'nonworkingset' tobedel|cut -f2 -d:|awk '{sum+=$1} END {print sum}'


The first two commands take care of getting just the list of numbers. Note that if capitalize sum then you should do it uniformly across the entire command.

octave

I finished coding an important segment of the C++ program, which is part of my research. The following are the tips I learnt:

1) Octave has strcat and strsplit functions in addition to plethora of other string handling functions. More here and here too

2) To save variables in a file and eventually start appending to it:
save('fname','varname','-append');
The single quotes are important. fname is the file name, varname is the variable name and the last switch is obvious. Octave does a nice job of saving variables with information like name of the variable, its size etc.

Blog visitor information

I experimented with

1) statcounter
2) sitemeter

for collecting information about visitor who visit my blog. Both offer free as well as paid version of their widgets. For better usage of those widgets, a good knowledge of javascript/html scripting would help a lot. I am planning to experiment with widget from Feedjit in future. Will update this post once I get to know more.

Novice bloggers like me need to read Google Webmaster tools page if they are interested to make their blog visible in searches.

two-dimensional array using C++ STL Vector

To create a two-D array of doubles using C++ STL templates:

I used vector of doubles as follows:

vector< vector <double> > arrayname;

I got this tip from yolinux. As can be seen, it is a vector of vectors. C++ STL templates come with a cost of cumbersome access to elements.  Please bear in mind that random access is not easy with this.

If you want to do some matrix manipulation, I would suggest you declare a two-dimensional array using pointer to a array of pointers as follows (here, val is the arrayname):

double **val;
val = (double**) malloc(sizeof(double*) * listsize);
val[0] = (double*)malloc(sizeof(double)* listsize * numcols);
for (i = 1; i < listsize; i++)
val[i] = val[i-1]+numcols;

Basically, we dump all elements to the location pointed by first pointer in the array of pointers and make our life easier for ease of access to array elements by setting the other pointer locations in relation to the first one. When you don't do this, you will have to map two-d index values onto one-d. Don't forget to free the pointers after done with using them (in order):

free(val[0]);
free(val);

Thanks to Numerical Recipes in C, I learnt the correct way of declaring and allocating two-dimensional arrays in C/C++.