Came across an article today linked to off the FreeBSD site that had some particular interest to me. I've had a lot of people tell me that Linux works better for database because of its supported filesystems. Well not for much longer!

"New performance improvements in FreeBSD 6 will take advantage of the new SMP architecture. It will also have the ability to scale to eight, 12, and 14 processors. The filesystem is now multithreaded, which, according to Long, will allow for much better performance of mail servers and database servers."

Which is awesome to hear, better SMP support and multithreaded file systems! FreeBSD community never ceases to amaze me. The article also talked about some neat features coming in the new version of OpenBSD which should make it an even better platform for BSD based router / firewall.

the full article can be found here: Return of The BSDs

Well the past 8 weeks of my life have had a lot of ups and downs I must say but it feels like its all finally paid off. I got to launch a new site on Friday that I'm really happy with. I got to work with some great team members and we all pulled together and created an awesome site, or at least I think so! Club SOAPnet launched on friday without a hitch. There are a couple of small things missing and should be a nice easy week of clean up. I spent a lot of time before beginning development working on the data model and looking through PEAR to see what packages will help me out. Some of the challenges were integrating with the Disney Global Registration system but that worked out in the end using curl to keep a seemless login process. It was nice not having to worry about storing sensitive data for once thanks Disney for taking that off my shoulders. I coded the whole site in about 5 weeks time using PEAR and my own custom, makeshift framework. My PHP framework isn't quite there but I'm very happy with the progress I've made on it with this project and it gave me a lot of insight into some more ideas I want to build into the framework. Hopefully if i get all that information gathered together outside my head I will share it with you all. Until then enjoy the site and let me know what you think!

So about a month ago we put in a new hosting infrastructure and me as the sole admin needed an easy way to monitor and keep an eye on all of my servers and the services and resources of each server.



So after some digging around I found that an application I use to use was now a new application Nagios. Now this application is great, took me about 2 days to set everything up as it is now, but once I found everything i needed it was great. Nagios uses SNMP Protocol to monitor almost anything on your server. It has a very OO configuration allowing you to reach some pretty complex setups. Mine didn't get too complex so i can't speak to all of that but it was pretty easy to setup once I got the hang of it. After I found these two Nagios Plugin pages I knew I was truely in heaven. NagiosExchange and Manubulon Nagios Plugins.



Nagios does a lot for me. It watches my partitions and lets me know if one gets close to full and notifies me before it gets full, watches RAM / Swap usage, checks to see that Apache, MySQL and SSHD are up and running, and even checks the status of my mysql replication to my backup server.

Though nagios was great, it didn't completely satisfy all my needs. Though it watched to make sure everything was up and running I couldn't see trends in usage and such. So I went out and installed MRTG on my monitor and began setting up MRTG stuff to log usage of network connections, cpu, ram, and hd usage on all my machines. This helps me to better see when I may need to upgrade things hopefully before even Nagios knows.
MRTG Page

All in all the key to a happy administrator is preparedness. Trying to keep this in more of a proactive environment than a responsive one. If anyone has any more questions about either of these apps let me know I'd be happy to answer some questions!

one of my coworkers sent me this link today, haven't had a ton of time to play with it but looks pretty sweet.

WWW SQL Designer

Which is a nice interface to GModeler that Grant Skinner made which runs in flash. Gmodeler seems a little more geared to software architecture than db modeling. Both fun and handy little tools

GModeler

A while back I wanted to test the ARCHIVE Engine in MySQL 5 on FreeBSD, well I had to do some jinkying around with the Makefile to add the options i needed to turn on the ARCHIVE Engine. I haven't seen it committed to the port yet so here it is:

# diff Makefile Makefile.orig
68,71d67
< .if defined(WITH_ARCHIVE_ENGINE)
< CONFIGURE_ARGS+=--with-archive-storage-engine
< .endif
<
150d145
< @${ECHO} " WITH_ARCHIVE_ENGINE=yes Enable Archive Table Engine Support."

After getting it compiled i ran a series of tests to see how it was working and to check the speed and how it compared to storing my data in a MyISAM table. Here is some of the stats...

mysql> SHOW TABLE STATUS LIKE 'campaign_16_long_myisam';
+-------------------------+--------+----------+
: Name : Engine : Rows :
+-------------------------+--------+----------+
: campaign_16_long_myisam : MyISAM : 28224724 :
+-------------------------+--------+----------+
1 row in set (0.00 sec)

mysql> DROP TABLE IF EXISTS `campaign_16_long`;
Query OK, 0 rows affected (0.02 sec)

mysql> CREATE TABLE `campaign_16_long` (
-> `id` int(11) NOT NULL,
-> `timestamp` bigint(20) NOT NULL default '0',
-> `ad_name` varchar(25) NOT NULL default '',
-> `type` varchar(25) NOT NULL default '',
-> `referer` varchar(100) NOT NULL default '',
-> `path` varchar(255) NOT NULL default '',
-> `client_ip` varchar(15) NOT NULL default ''
-> ) ENGINE=ARCHIVE DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.02 sec)

mysql> INSERT INTO `ad_tracking`.`campaign_16_long` SELECT * FROM
`ad_tracking`.`campaign_16_long_myisam`;

...
# top
last pid: 85206; load averages: 0.99, 0.74, 0.37

up 8+12:37:29 03:02:10
66 processes: 2 running, 64 sleeping
CPU states: 25.0% user, 0.0% nice, 1.1% system, 0.0% interrupt, 73.9% idle
Mem: 113M Active, 3212M Inact, 184M Wired, 158M Cache, 214M Buf, 5368K Free
Swap: 2048M Total, 2048M Free

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
85044 mysql 20 0 59756K 34092K kserel 1 6:52 98.10% 98.10% mysqld
...

Query OK, 28224724 rows affected (8 min 42.90 sec)
Records: 28224724 Duplicates: 0 Warnings: 0

mysql> SHOW TABLE STATUS LIKE 'campaign_16_long';
+------------------+---------+---------+------------+----------+
: Name : Engine : Version : Row_format : Rows :
+------------------+---------+---------+------------+----------+
: campaign_16_long : ARCHIVE : 10 : Compressed : 28224724 :
+------------------+---------+---------+------------+----------+
1 row in set (0.00 sec)

# ls -alh : grep campaign_16_long
-rw-rw---- 1 mysql mysql 19B Sep 16 02:54 campaign_16_long.ARM
-rw-rw---- 1 mysql mysql 508M Sep 16 03:04 campaign_16_long.ARZ
-rw-rw---- 1 mysql mysql 8.6K Sep 16 02:54 campaign_16_long.frm
-rw-r----- 1 mysql mysql 2.3G Sep 15 19:00 campaign_16_long_myisam.MYD
-rw-r----- 1 mysql mysql 249M Sep 15 21:35 campaign_16_long_myisam.MYI
-rw-r----- 1 mysql mysql 8.6K Jul 7 13:57 campaign_16_long_myisam.frm

The server moved 28,224,724 in 8 min 42.9 seconds. It compressed the
data from 2.3G to 508M AMAZING storage saver! I am truely happy,
running selects off it were just as fast as the MyISAM:

mysql> SELECT * FROM `campaign_16_long_myisam` LIMIT 0, 30;
...
30 rows in set (0.00 sec)

mysql> SELECT * FROM `campaign_16_long` LIMIT 0, 30;
...
30 rows in set (0.00 sec)


System Specs:
FreeBSD 5.4-RELEASE-p7 #0: Wed Sep 7 14:12:34 CDT 2005
CPU: Dual Core AMD Opteron(tm) Processor 265 (1792.85-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3

4096MB Ram
2 x 146GB SCSI/ RAID 1

you can read the full thread from the mailinglists here

All in all its a great storage option especially if you're storing lots and lots of data. The fact that you can only SELECT and INSERT is nice to, for stat logs and such its ideal and you get a little more peace of mind knowing your data is a little safer now!