This is a nice short example of how to use SAX to read in XML files in Java.  I’m sure I’ll be referencing this page sometime in the future.

So, at work, a portion of a program that I am working on has to perform some cleaning procedures on just over 200,000 records.  Cleaning each record iteratively was taking way too long, so I decided to use some of my parallelization skills from last summer to see if I could decrease the run time on the cleaning portion. I decided to use 4 threads to ease the work on our poor processor. Here are some of the results.

Cleaning 5,00 records:
Iterative Clean Average Time: 1.1864 seconds
4-Threads Clean Average Time: 1.3628 seconds

Cleaning 50,00 records:
Iterative Clean Average Time: 209.28 seconds
4-Threads Clean Average Time:  39.23 seconds

For the smaller number of records, the threaded clean performs slower, but for the larger number of records, there was a serious increase in performance.  I then wanted to measure just how much of a performance gain I acheived.

If you perform the following calculation:

(100) * (39.23 / 209.28)

That says that for 50,000 records the threaded cleaning procedure runs in in about 18.74% of the original time.  This percentage was a bit shocking, as with 4-threads, I should only expect each to take on 25% of the work. Perhaps, I needed to time more trials (I only did 3 for each), but either way, I think I’m going to go with the parallel cleaning procedure.

So, at work, I’ve been trying to migrate all my existing work, projects, services, etc.. from a Windows Server 2008 machine to a fresh install of Natty Ubuntu.  Developing on windows, even with cygwin was getting to be too much of a hassle for me.

However, when I went to migrate the MySQL database, I ran into a minor difficulty that took me nearly a full afternoon to figure out.

I started by running a simple mysqldump command on the windows machine

cd “C:\Program Files (x86)\MySQL\MySQL Server 5.1\bin"

mysqldump.exe -u root -p databaseName > databaseName.sql

I quickly did an ‘scp’ to the Ubuntu box and then ran the following commands:

mysql -u root -p databaseName < databaseName.sql.

MySQL then told me the file contained a syntax error.  This seemed impossible, as it was generated by the MySQL dump command on the windows server. I assumed the problem had something to do with the formatting of the text file (e.g. those stupid ^M characters).

What fixed the problem however, was running a refresh command BEFORE doing the dump.  The problem-solving, fix-all command is below:

mysqladmin.exe -u root -p refresh

I’m getting a 4ft by 3ft print of this. Yeah, I’m excited. :)

Screen Shot of my workstation at U Maine.

For those who are closely following my experiences here in Maine, I apologize for the delay in this post.  Hopefully, after reading this, you will understand why I did not post the day immediately following the first.

June 1st - Orono, Maine.

 Today is the first day of the program and we are to meet in the Barrows Auditorium at 9am.  We were told yesterday that breakfast, lunch, and dinner will be provided and that it takes about 10-15min to get there.  So after getting up, doing some push-ups, and a quick shower, I meet Brianna, Katherine, and Gerry in the common area downstairs.  Thankfully, one of the DTD (Delta Tau Delta) brothers is headed in the general direction as us, and he agrees to walk with us for most of the way.  We make it to the auditorium without a problem (save the rain), to find the other 6 participants.  On the back table there is a huge array of food, including: croissants, Costco muffins, coffee, orange juice, and entirely more fruit than necessary.  We eat breakfast and socialize for a while and then Yifeng (one of the two program directors) calls us over to sit and things begin.  I can already tell that he will become a very familiar face this summer.

Yifeng goes through some of the more technical logistics of how things are going to work this summer.  We also fill in and sign a bunch of forms to say that we will be employees of the University of Maine -most of the morning consisted of technicalities.  I’ll save you the details.

Before you know it, lunch time has rolled around, but I am not entirely hungry because I had absolutely stuffed myself full of strawberries, grapes, cherries, and other fresh fruits I have lived without at DU for so long.  Nonetheless, 12 boxes of pizza soon arrive, and I am destroying slice after slice.  The pizza has a serious small-town taste to it, and I can’t seem to get enough.  After we have eaten lunch and talked some more together, the faculty arrive.

This is where the REU program really starts to get interesting.  7 professors all from different departments gave power-point presentations on which project we would be able to choose from later.  The first presentation is given by Bruce, the other program director.  Bruce is a jolly, balding, old fellow, who has the personality of both a computer engineer and Santa Claus; trust me, this guy is awesome.  Regardless, his presentation involved the visualization of data at high-resolution.  Basically, he wants to build a screen as big as a projector screen using a grid of 16 monitors.  Using monitors, instead of a projector, allows you to have good resolution from afar and close-up, in that you don’t have to zoom in on anything.  Being a foot away from a 4x4 monitor-screen is just like being a foot away from any normal computer screen.  The resolution possible on one of these screens in nuts.  Example from google images.

The other 6 professors also gave presentations, Yifeng included, but also gave off more serious vibes.  Prof Tom gave a presentation on magnets, and Andre talked about Bio Medicine.  All of the presentations involved some sort of use of super-computers, in that they have ALOT of data that needs to be processed and the algorithms needed to process this data are not at all simple.  The sort of processing power needed to handle the problems they presented could only be taken on by a super computer. 

With regard to what they presented, it was all very technical.  What I gathered is that the REU students will be writing code for a variety of professors of different fields (physics, bio med, marine science, climate change, etc…) who know their appropriate subjects very well, but are unable to write much code.  So, basically, as an REU student this summer, I will be contributing to research that furthers the field of, say, physics.  Some of the professors even mentioned that they plan to publish their research findings sometime during the summer at an international level.  Yeah, I’m excited too.

After sitting still for several hours, I’m nearly ready to pass out.  To find out why, read what happened in my last blog.  Thankfully, I’m not entirely alone.  Yifeng realizes that many of us are getting sleepy, as we’ve been in this room all day.  We carpool are way out of Barrows and arrive across the river to where the supercomputer is located.

By the way, the University of Maine, is located in the small town of Orono in central eastern Maine.  A beautiful river runs through the town, and right by the university.  In fact, its about 40 yards behind my fraternity house.  Follow the road on the right nearly all the way to the top of the photo, and I live just near the river.

The super computer is located in a community center that allows businesses to come in and use some office spaces, meeting rooms, and other generally boring things.  So, inside this building, there is one entrance to the super computer that requires a magnetic key card for entry.  There are 4 exits to the room, but none of them have handles on the doors back in.

To my surprise, I learn that most of the people who use this supercomputer have never seen it in person.  In fact, not that many people, aside from maintenance, the creators of the computer, and REU fellows have even been this close to it.  There are two main racks that contain all the different nodes, or mini-computers.  All 128 of these nodes are connected by both fiber optic cables and ethernet cables at two of the biggest switches i’ve ever seen. In between these 2 racks of dual core mini-computers, each as fast as my laptop, is a central control station.

Interestingly, the super computer seen in these photos replaced this one.  The computer you see here, is obviously, just a bunch of computers wired together to generate some beastly processing capabilities.  It used to be in the top 500 super computers in the world at one point.  Now, it’s a bit dated.

After touring the super computer building, we all headed out to dinner.  Yifeng said dinner was payed for through the program, so I ordered a coke and a rib-eye steak.  Dinner was a little awkward because none of us really knew each other yet, and well.. we’re all computer geeks. Nonetheless, we took a pretty good picture.

Finally, we all got back to our respective homes, a little exhausted from the long, yet amazing 1st day.

This post will hopefully be the first of many. I am writing with the intention of informing those whom I care about of my experiences here at the University of Maine. So, please, sit back, relax, and continue to follow my Comp-Sci Saga.

For those of you who do not know, I was accepted into a summer-internship program called “Research Experience for Undergraduates” (shortened to REU) located at the University of Maine. The program is financially sponsored by the Department of Defense (DoD) and the National Science Foundation (NSF) so that undergraduates, such as myself, will have a chance to get hands-on, real, research experience. But before I explain how technical things are becoming, allow me to rewind, and describe the last 48 hours.

May 30th 3:45pm MST

I have been packing my room since 10am and I have not eaten lunch. It is at the point where everything I need for Maine is in 2 bags, and everything else in the room must go! I am running around my room, my kitchen, and bathroom finding the last bits of everything and putting them into the last box I have available. I was supposed to be completely out of my room 40min ago. Finally, after every last item is in the box, and all the furniture is wiped down, I make 3 runs to and from my car with arm-loads of items. At last, I call my R.A. Alyss, and she makes her inspections, we chat for a bit, and she walks me down to my car, both of us carrying the last of my things. We make our goodbyes, and I drive away from Centennial Towers, knowing I will not be returning into the building for a long time.

May 30th 4:45pm MST

I then make my way to pick up a newly made friend Allison, as we had planned earlier to meet and talk over Jamba Juice. After a fairly short, but pleasant visit, I realize I’m running low on time, so I decide to drop her off, and then take my things to the storage unit. Thankfully, unloading my car by myself wasn’t too terrible as there were push-carts available. The whole processes of taking things out of the car, bringing them up the elevator and then packing them again took around 40 minutes. All the while I am making phone calls to my best friend-girls Sara Snyder and Abby Johnson, trying to figure out my schedule for the day.

May 30th 5:30pm MST

I am done packing and am left with 2 bags ready for Maine, and a stack of blankets in my car. I had planned on mailing them to myself, however, after talking to my parents, I realize that this plan is impossible, as all shipping locations are closed on Sunday. Instead, I improvise and buy a large duffel bag at Target. Here is where things started to get a little crazy. I am trying to meet Abby at 6:30pm so we will be able to go Tango dancing, but I also need to say goodbye to Sara, in person. Sara is a 15min drive from campus, and it is now 6:00pm. I haul ass to the mall where Sara and her boyfriend, Rob, are located, and we talk over me scarfing down a slice of Sbarro pizza. 10 minutes pass faster than I thought possible, and I am again back on the road, this time, holding back some tears, realizing I’m not going to see my best friend for quite some time.

May 30th 6:40pm MST

I am late for when I told Abby I would meet, but thankfully, she’s not doing a whole lot and is still on campus. I park and lock my car for the summer, and Joel, Abby’s boyfriend, helps me load stuff into their car. The 3 of us drive to Cheesman Park (picture from google images), where day 3 of the Denver Tango Festival is going on. There were about 200 people dancing inside this glorious gazebo. Imagine this dance floor crowded to the point where you’re less than 1ft from any other couple dancing, throw in a beautiful sunset, and 2 great friends, and you’ve got a glimpse of my evening.

May 31st 12:00am MST

I am exhausted. The day, as mentioned above, has been non-stop since 10am, and I have yet to eat anything except that slice of pizza from Sbarro. Yes, a Burger King was open on the other side of security, but my flight boards in 10min, and there is no time to make it back. Not even a convenience store was open for me to buy water from. I briefly talk with my family on skype, but all I want to do is pass out due to my hunger, busy day, and crazy night before (details not mentioned here, but let’s just say it was my last night at college as a sophomore). Headphones in, ready to collapse, I board the plane.

May 31st 6:20am EST

I am now drifting in and out of sleep, in a chair, outside the gate of my next flight in Philadelphia, PA. Attempts at sleep during my flight from Denver were futile as I have been situated between two decently large men for the last 6 hours. In 40min, I can eat food (as the shops open up) and in about 2 hours I will fly to Bangor, Maine. The aggregate of all 15min sessions of sleep I managed throughout the night (and this morning in the chair) sums to about 100 minutes. A small coffee, croissant, and water bottle later, and I’m on a plane to Maine.

May 31st 11:00am EST

Bangor International Airport is one of, if not the, smallest airports I have ever been in. How they claim to be an international airport, I do not know. After passing the security barrier, I checked the electronic reader board to find which baggage claim my thing would be at, but then I looked up and realized there were only 2 claims to choose from. Upon looking up, I also realized that Sara’s descriptions of “Maine-ards” is correct. She noted that not many people in Maine have much of a sense of fashion, and the airport served perfectly as an exemplar, showcasing a variety of outfits, most including sweatpants. I call Yifeng, the REU program director, to ensure that I still have a ride from the airport to the university. Robert, a graduate student arrives a few minutes later, asking me, “you look like you’re waiting for someone?”

May 31st 12:00am EST

I say goodbye to Robert and step foot into the Delta Tau Delta fraternity house. Here is kindof a cruddy photo of the house to give you a sense of what it looks like. I’ll try and get a better picture up when I get my camera. I look around the living room, not sure who to talk to or what to say, but with Robert gone, I introduce myself. Shortly thereafter, I meet Bill, who remembers that the REU students are moving into the house today, and gets some brothers to clean up. About 25min later he presents to me a room, now with a desk, bed, and a shelves in it. I spend the next hour un-packing my things. This is where things finally start to calm down.

May 31st Afternoon EST

Eventually a car pulls up in front of the DTD house, and a girl and her mother are soon in the lobby downstairs. After her and her mom make a trip or two up stairs with her things, I introduce myself, and offer to help carry stuff up; we live on the second floor. Once everything had been taken up, Katherine and her mother offer me a ride to the grocery store, and I eagerly accept. Recall that the only things I have eaten since 10am the previous morning have been: jamba juice, a slice of pizza, a croissant, and a small coffee. I buy some necessities for the room, and fridge. We return with plenty of groceries, and I eat nearly $15.00 in snacks.

May 31st 5:30pm EST

Finally there is some food in my stomach and I have taken a 40min nap! I am temporarily refreshed. By this time, I have met most of the brothers living in the house for the summer (there’s about 8 to 12) and we’re all getting along quite well. Being a fraternity member myself, we all quickly bond, and I am invited to come to a Chinese Buffet. The buffet is not of the best quality, but food never tasted so good. I destroyed 2 full plates of noodles, beef, chicken, fruit, etc… The 8 of us all pay on separate bills and return home.

May 31st Nighttime EST

There are now 4 other REU participants in the frat house. Katherine, whom I met earlier, just finished her freshman year at RIT (Rochester Institute of Technology) and is a volleyball player. Brianna, a soon-to-be senior CS major is from Southern Maine, and is pretty quiet. Gerry, is in his 30s, has his B.A. in theater, and just recently decided to return to school for a B.S. in computer science. Conversation is a little odd between the four of us, but we bond pretty quickly, realizing we’ll be spending the next 10 weeks together as neighbors. The frat brothers decide to “celebrate” veterans day and everyone winds up hanging out between rooms, in the hallway. Before you know it, a game of beer pong is started, and everyone’s having a pretty good time. I play two games of pong and a card game called “asshole.” The brothers are hilarious, and everyone seemed to have a great night socializing. At last, I find 8 hours of solace in my bed.

part 2 to be written tomorrow…

After browsing through programmingforums.org I found someone who needed to know about the different normal forms in databases.  My response to her question is copied & pasted below.

Here are descriptions of the normal forms. Sometimes it’s easiest to understand the forms by seeing what violates them.

1NF - Typically, if you can make a valid database table, it’s in 1NF. I think technically the table should have a unique key to be truly in 1NF

2NF - A table is in 2NF if and only if none of its non-prime attributes are functionally dependent on a part (proper subset) of a candidate key. (A non-prime attribute is one that does not belong to any candidate key.) Basically, you can’t have any partial dependencies. An example of a 2NF violation would be…
AB -> CD, B -> D. AB is a candidate key, B is a proper subset of AB, and B implies D, which is a non-prime attribute: a 2NF violation!

3NF - For a table to be in 3NF, it must first meet the conditions for 2NF. Also every non-prime attribute of the table must be non-transitively dependent (i.e. directly dependent) on every key of the table. That is there can be NO transitive dependencies! A transitive dependency is a type of functional dependency in which a non-prime attribute is determined by another non-key field and that field is not a candidate key.
Possibly the most simple example of a 3NF violation would be…
A -> BC, C->A. A implies B & C, C -> A so C must imply B by transitivity. Transitive dependencies violate the 3NF form.

BCNF - For a table to be in BCNF, it first must meet the conditions for the 3NF. Also, a table is in BCNF if and only if for every one of its non-trivial functional dependencies where X -> Y, X is a superkey—that is, X is either a candidate key or a superset thereof. In plain English, any attribute(s) that imply any other attributes must be part of a key.
Here is a simple example of a table in 3NF, but not in BCNF.
A -> BC, B - > C. A implies both B and C, and B implies C, but A does NOT imply C by transitivity, because it implies C trivially.

This weekend I had a lot of extra free time and I was feeling a bit under the weather, so I decided to make use of this time that I couldn’t be out with friends (or out shredding the gnar on my snowboard) by installing Ubuntu Linux 9.10 on my Mac.  I encountered a variety of problems getting everything to work, so here’s how it went:

1.) Start up Boot Camp Assistant.  This can be found in /Applications/Utilities/  — For those of you who still don’t have leopard (10.5+) you have to download Boot Camp Assistant. Use google to find it.

2.) Next, click Continue. I figure printing the Installation and Setup Guide is for sissies!

3.) Allocate some space for Linux.  Do this by dragging the slider on your hard drive.  I gave Linux 15GB and left the rest of my 250GB for OS X. Here is where I encountered my first ERROR!

Formatting my hard-drive, and restoring it from a backup seemed like a scary idea, so I went to Google for some alternatives.

Defragmentation in Mac OS X: I found out that one can get the files to move by defragmenting the hard-drive before trying any partitioning.  Supposedly, the combination of how OS X is written along with their journaled file system, should extinguish the need for a defrag.  However, after running iDefrag, my system definitely needed it.  Unfortunately, all the defrag programs available for mac (after a very short Google search) cost money.

Defrag via Time Machine & System Restore: I gave up searching for a defragger and decided to take my Mac’s advice and do a system restore.
A.) First, I updated my external usb hard-drive (for backups) using Time Machine.
B.) Second, I plugged in my Snow Leopard cd and told my computer to restart (keeping my Time Machine Backup HD plugged in).
C.) To boot to the cd, during start up, hold the Alt/Option key.  Eventually bootloader in the form of a picture of Macintosh HD and a CD will pop up.  Click the CD to boot to the cd.
D.) Once the computer has loaded everything from the cd goto Utilities > Restore from Time Machine Backup.

About 4 hours later, my issues with partitioning had been fixed! Returning now to Boot Camp Assistant…

4.) The next screen (after the partitioning is complete) in Boot Camp Assistant asks you to either “Start Installation” (of windows) or to “Restart Mac OS X” — Since we don’t want to install windows, this is where you plug in your Linux Installation CD and click “Restart Mac OS X”

5.) Boot to the cd.  This can be done (as mentioned earlier) by holding Alt/Option during startup.

6.) Proceed to install Linux as you normally would, until you get to the partitioning section. Choose to set up the partitioning table manually. Boot Camp should have allocated your space for Linux in the form of FAT32, leaving about 500MB of free space between Mac and the Fat32 space.  Go ahead and delete this partition.

7.) Now, I’m not really sure why they leave the free space, but I figured it should stay there to be safe.  Since you now have a giant chunk of free space, it might be difficult to allocate both space for “swap” and “ext3.”  I achieved this by using the following equation:

ext3 Linux Space = Free Space - (500 for the barrier + 1024 for swap)

Create the ext3 allocation at the END of the hard-drive, and then do the same for the swap.  The mount point should obviously be ‘/’ in the ext3 partition (Note: keeping swap closer to the center of the hard-drive gives it quicker access and therefore gives you a slight performance increase when actually running linux) If you did everything right, your hard-drive should now look like

Mac OS X, 500MB Free Space Barrier, 1024MB Swap, Ext3 Linux.

8.) Finish the installation and reboot, taking the cd out.

Your computer is now ready to dual boot to both Mac OS X and Linux!! Whenever you want to boot to Linux, just hold Alt/Option during start up, and choose “Windows.”  You’ll be taken to Linux’s boot loader (either GRUB or LILO).

Finishing Notes: For more information regarding Mac’s journaled file system structure check out this link. http://support.apple.com/kb/HT2355.  Also, I installed Ubuntu 9.10 onto an ext4 partition, which worked just fine, and I imagine, is better than ext3.

Today I’m starting a blog about my findings in computer science, or just with computers in general.  That is, once I figure out how to do something noteworthy on (or for) a computer, I’ll post here with instructions and details with regards to how I did it. So, sit back, relax, and enjoy reading how I further my persona toward “geek.”