When Nothing Is Something

Dear Diary,

I decided not to come into the office this weekend.  It was awesome!

But it meant that I hadn't run any calculations that I was supposed to.  That was mostly because I couldn't figure out how to compile the program Miles had written.

So this morning I was informed by Steve and Miles that I should switch to a different branch.  Huzzah!  It all compiled!  Now things are running...

Just another example of how doing nothing can still be productive.  But now I have to be really productive.

Xcode

Dear Diary,

Stefan showed me a way to manage of a lot of code.  More specifically, Xcode or Eclipse with the former being only for and required on Macs.  What this handy bit of software does is to sort through all of your programs that you're writing.

The most useful thing I've found so far is that if you right click on a function you've defined, you can have the program show you where it is defined.  Neat!  I always looked down on Xcode when I programmed earlier because it was the default for opening program files, but I see it has more use than just making apps and being slow!

Soft Linking

Dear Diary,

I have two folders that I want to access the same information.  Like, let's pretend I have a pdf of something totally swanky that I put into one folder.  For my own filing system, I want to have another folder with a different name that accesses the folder with my swanky pdf.

All I have to do is soft link!  By typing

"ln -s [original directory] [new directory]"

the computer makes a folder that if you go into it, you go to the original folder!  This also works with documents!  I just made one for our journal club tomorrow so I don't have to move all the documents over.  I use them all the time if I see two projects need the same thing:  I soft link both to the relevant whatever!

Our little in-house server has a lot of these.  Many times, students have tried to go wandering around and document the directory tree, but few have made it out!  There's so many soft links everywhere that it's turned into a maze rivalling Crete and only now generates headaches!  But no Minotaur (I think...)!

In fact, the streets of Irvine and many foreign countries such as cities in Spain are set up like this.  The idea is that if you have windy streets, then you can get away from bad guys more easily by running until you're lost yourself.

Mucho Macho Pruebas!

Dear Diary,

The cluster has been humming with my calculations recently.  I'm preparing lots of data for Li.    I've spent a lot of time debugging how I was making all 2000 points for him and found one key error:  the number of spaces between four fingers is three (not four).  Basically, I set up my code to automatically generate the program files somewhat strangely.

Anyway, now I'm pumping out the correct data and will send it to Li as it comes out.  The cluster really makes it slick to calculate large data like this and I can't even imagine doing it on my personal machine.  I can basically run things overnight!

I have noticed that copying the scripts to set up all of the programs is really quick with Unix to boot. So, giving them more data shouldn't be a problem if it comes to that.

Now, Kieron asked us to run a "macho test" and I haven't quite looked into that yet.  Hopefully, I can see if something is located in previous papers, so I'll go take a peak now.

Cluster B(l)uster

Dear Diary,

Well, there's no easy way to say it...but I suppose you should know:  I broke the cluster.

For several weeks, those who use the computing cluster have complained that it's been abnormally slow.  In fact, I even received an email from another graduate student concerned that I might be using the resource improperly.  I denied it at the time, but it appears that I was complaining about myself for nearly a few months.

In retrospect, I should have known that something was wrong because the data was sometimes coming out strangely.

You see, you submit jobs to a temporary directory where it submits everything to the cluster machines.  If you need to copy back a lot of files from the temporary directory, then it gums up the system.   And I'm guilty here.  To fix it, my little script submission file now only has ">out" instead of ">(home directory)/out" to prevent it from copying everything back.  Further, I only needed two files out of the possibly thousands of files that are output.  So, I just added 'make clean' to the submission script and that gets rid of everything before it comes back to me.

Charge 2 is requesting the correct number of computers and threads.  I had requested one computer and one thread, but a command for OpenMP (to parallelize the code and make it go faster) was set to request 8 threads.  So, my program was unhealthily bumping up against other's corrupting everyone's data and making it so that writing files out was not working.  This really made things messy.

The key here is to not look at the output files.  It says, simply, "nodes".  I had requested 1 node (aka computer) with ppn=8 (for 8 threads), but it reported 8 nodes...So, I should trust I filled out the fields correctly instead of looking at the output file...Summarily: nodes=1:ppn=8 (for 8 threads).

Now, none of this is particularly egregious...and I would have gotten away with it (if it weren't for you meddling kids!) but I was running so much data that little mistakes made a big difference.

I'm going to go and barricade myself over the weekend and lay low until this whole thing blows over...oh, and I'll run that data that I owe Li.  These fixes actually come at a good time because running jobs over the weekend is best (less use the cluster).

Cluster Q&A

Dear Diary,

Just recently, the resident computer cluster ran out of memory and quotas were reduced.  I recently sat down with one of the users who had an enormous amount of data sitting on the computer at the time.


DrWho: So, what did it feel like to have your name on the list of people who would be over the new, reduced quota?
V: Uh, well, nothing much really.
DrWho: You must have been impressed with yourself.
V: I guess...really it's my fault for keeping so much data on the login computers.  We have a whole separate server for things like that.
DrWho: Did you lose any data?
V: No. In truth, I hadn't run anything for a month or so.  Did you?
DrWho: Yes, I was running some things the night before, so I must have put the server over on it's gross quota before they reduced everything.
V: Oh, oops. Bummer. Hopefully it wasn't important...
DrWho: No, of course not.  I do have to somewhat annoyingly truncate my enormous calculations with the reduced quota...What advice would you have for incoming graduate students so they don't overuse their resources?
V: Why not just ask for a bit more quota space?  Wait, this isn't going on your dumb bl-
DrWho: -and we'll have to keep it there for now.  Thanks for the interview and tuning in!

In the Double Blind

Dear Diary,

I finally got the data to run after hitting a few snags.  The cluster really does wonders with computing time!  I was able to run about 200 runs overnight by writing shell scripts to automate the submission to the queue.  Before, I'd just run it on my computer and it took about a week or more depending on the system.

From the sampling of data, I took ten sets and sent them to Li to make this quasi-double blind.  He'll take the data and try to reproduce the curve I have made with the whole data set.  We'll see what happens!

Now, I suppose that Li could waltz on into my office with a package of dark chocolate imported from Belgium and suggest that I hand over all the data before I'm supposed to, but there's all this scientific integrity thing. Drat.

Kieron also asked me for a Christmas present.  I am to determine the entire plan for the remainder of my time here.  However short (!).  This should be fun!  I can tell him all about the big plans I have!

Post Thanksgiving Rush

Dear Diary,

Thanksgiving just ended.  I had a lot to eat and didn't do much work (don't tell Kieron!).  It was good for my focus to take a short break.

Professor Burke has become my enabler, sort of, by continuing to hook me up with great projects. Namely, I have to run some data for Li.  I started running the necessary components on the computing cluster but messed up a small file issue, so I fixed that today and will send it soon (it doesn't take very long to run on the cluster).

The catch is that I have to arrange this so it's single blind.  Li can't know the whole data set, and I need to feed it to him piece meal.  It should be exciting!  I'll keep you posted!

Before the end of the calendar year (i.e. winter vacation), I'm going to try and work fast to get a few pesky tasks completed.  I should get to it...

E tu, Macro?

Dear Diary,

Oh, Lucas!  You left me with an unfinished paper (thank you, sort of).   Normally, I would be thrilled to help you get another publication, but you have so many macros in your Latex documents to make things so much simpler to type up.  I'm from the school of type everything out long hand!

So, I made a new figure and then tried to implement it into the tex file.  You know

\begin{figure}
\includegraphics[width=\columnwidth]{newpic.pdf}
\caption{Figures.  It's a figure. \label{fig:itsa}}
\end{figure}

but then it didn't compile!  I tried and tried (I even removed the above to see if it worked and it did!) until I removed a different figure and somehow it compiled!  It turns out that one of the macros later on was busted and this mysteriously revealed itself when I added a new figure.

So, there!  Don't use macros!  They only make your life easier!  I should go tell Justin to stop using macros to make changing symbols easier if he wants a switch and just go do everything by hand every time he makes a switch!

Obey your Master(.bib)

Dear Diary,

Wow!  I never knew I could be that productive!  I just successfully eradicated all traces of malfeasance in the group's Master.bib file by spending two hours straight parsing errors!

Master.bib is the Bibtex file that the group uses to enact it's evil plot to collect every reference that's ever been referenced! (Current unique references: 2344)   Maybe I already explained this to you, diary, I can't remember because my head hurts...

Anyway, it appears that much more head hurting occurred in previous generations of Burke students.  The file wouldn't compile for a myriad of reasons:

1)  "%" signs must be listed as "\%" for Latex to work
2)  Oh, Lucas!  You can't press return in the middle of an abstract! Latex thinks it's a new line!
3)  Curly parentheses are critical
4)  The numbers 1998 and 24 are completely different.  Even scarier: Latex knows the difference between a year and a TV show.  I have to make sure I keep that straight and not lose track of the date!  Oh no! It's daylight savings time apparently!

Anyway, it's done.  Enjoy, everyone.

Return of the Super Lucas

Dear Diary,

Dr. Lucas came back today!  Just like he said he would!

I gave him a quick review of what I've managed to do over the past few days (graphs, numbers, etc.).  Afterwards, he went onto his computer, fixed the inversions, DMRG, and our local computing cluster during my lunch break with surgical precision.  The project is now way ahead!

About that computer running slowly:  There was some hullabaloo about how slowly the main computing cluster was running this morning. Detective Lucas, Ph. D. did some grade A sleuthing and found the culprit was a Matlab program not running in the queue (always queue!). We identified the original suspect since his digital fingerprints were all over the file (exposed by the 'top' command).  This guy was a former summer member of the group, but his alibi was airtight! As it turns out, it was John the whole time! Yay, Lucas!

Lucas then went over to Ms. Jenny Du with all the souvenir receipts from his trip. One thing to keep in mind if we travel, Diary, is that you can get reimbursed for the miles you drive.  So I'll have to keep a log of that (with receipts!).

Lucas lucked into a neato trick to access a file.  He had a bunch of folders in a directory with only one that had "redo" at the end of it.  He typed cd *redo and it worked!  If the folder isn't unique, then it goes to the first one on the list...Ok, so maybe that's not that interesting.  The way I originally understood the problem and solution was completely thrilling, but I didn't have it right until Lucas helped me out!

Less time programming, more time lunching

Dear Diary,

In order to make things more mechanical and avoid my clumsy human error, I made some shell scripts to run the programs for different parameter values.  Everything now works after one click and I don't have to reset everything by hand each time I want to rerun the program.



You see, all the terminal commands that you would normally put into the terminal can be loaded in a text file (extension: .sh) that begins with #!/bin/sh so that when you run the file.sh (by typing ./file.sh), the commands are run in order in the terminal.  You see, diary, technology has made life better.

Working towards As while learning C(++)

Diary,

Classes have really been eating into my research time.  I successfully ran the codes that Lucas told me about.  I've decided to pick up on code while I learn what it means to maximize my working time.  They say that it's better to start working on something right away than read a lot of literature first.  That's because you can be productive while you pick up the finer points of the research.

The group likes to use C++, but my background is in Fortran.  There seem to be a lot of online sampler programs and books in the library that I want to use to learn C++.