00:07:18 *** yfttyfs (n=yf@221.223.131.45) has joined #pvfs2
08:06:40 *** |icebone| (n=icebone@pd9e374a0.dip.t-dialin.net) has joined #pvfs2
15:51:53 *** yfttyfs has quit (Remote closed the connection)
16:06:11 *** dmlb2000 (n=dmlb2000@71.93.15.147) has joined #pvfs2
16:07:51 <dmlb2000> Anyone alive that wants to help me get pvfs working on rhel5?
16:07:57 <dmlb2000> having problems mounting the client
16:08:28 <dmlb2000> followed the quick start for testing on a single computer and everything is working okay except for the mount of the client...
16:12:05 <dmlb2000> I get some 'unaligned access to <address>, ip=<address>' when creating the server
16:14:43 <robl> dmlb2000: the unaligned access should be just a warning, but we can get those fixed
16:14:58 <robl> dmlb2000: we don't have any rhel5 boxes to test
16:15:13 <robl> dmlb2000: what architecture is this?
16:15:18 <dmlb2000> robl: ia64 ;)
16:15:26 <dmlb2000> yeah you probably don't have that either ;)
16:15:47 <robl> dmlb2000: ok, cool.  pw does, but they are running rhel 2.1(!)
16:16:00 <robl> dmlb2000: ok, so pvfs2-ping works ok?
16:16:51 <dmlb2000> wow, okay... I don't remember if that app built... I hold up let me check...
16:17:48 <dmlb2000> Should I be building this with gcc34? or have you guys tested with the gcc4?
16:18:15 <robl> dmlb2000: we should be ok with every gcc, even as old as 2.95
16:18:26 <dmlb2000> sweet :)
16:20:05 <dmlb2000> Oh, have a quick question about use as well, does pvfs2 provide the user any sort of controls over where his files exist (on disk)?
16:20:26 * dmlb2000 comes from a lustre background mostly...
16:22:30 <dmlb2000> Okay pvfs2-server is running... and I thought pvfs2-ping is supposed to run on a mounted pvfs2 filesystem
16:22:37 <robl> dmlb2000: there are some, yes: http://www.pvfs.org/pvfs2-faq.html#sec:dir_tuning
16:22:57 <dmlb2000> usage says to do something like 'pvfs2-ping -m /mnt/pvfs2'
16:23:03 <dmlb2000> Thanks.
16:23:05 <robl> dmlb2000: we have a userspace-only option
16:23:54 <robl> dmlb2000: we can bypass the VFS-mounted file system if we have an entry in /etc/fstab or an /etc/pvfs2tab file
16:24:06 <dmlb2000> Oh, okay... I'll try that
16:24:09 <robl> dmlb2000: bypassing the VFS makes debugging a little easier
16:24:26 <robl> dmlb2000: and it's the interface we use for our MPI-IO driver
16:25:36 <dmlb2000> Okay, pvfs2-ping seems to think its configured...
16:26:23 <robl> dmlb2000: cool, now back to mounting
16:26:33 <dmlb2000> mount: Connection timed out
16:26:41 <robl> dmlb2000: do you have pvfs2-client running?
16:26:45 <dmlb2000> that's what I'm seeing on the mount command...
16:27:09 <dmlb2000> Yeah, that's the app that didn't build... any specific configure options to enable that?
16:27:15 <robl> dmlb2000: ah, ok
16:27:42 <robl> dmlb2000: did the kernel module (pvfs2.ko) build?
16:28:18 <dmlb2000> I had to do it manually but yeah I can build that easy enough ;)
16:28:53 <dmlb2000> I passed the --with-kernel= option to configure and the output at the end told me that 2.6 kernel was going to be built 
16:29:04 <robl> dmlb2000: cool.  did you run 'make kmod' ?
16:29:05 <dmlb2000> but I had to cd to the kernel mod directory and type make to build it
16:29:50 <robl> dmlb2000: 'make kernapps' will build pvfs2-client, but that target should get triggerd with 'make kmod'
16:30:07 <dmlb2000> Okay running that now...
16:30:24 <dmlb2000> didn't know about these targets... hmmm...
16:31:02 <robl> dmlb2000: our quickstart is adequate, but could use a good revising
16:31:23 <robl> dmlb2000: it's in section 5.2
16:31:46 <dmlb2000> especially if its anything beyond just a simple './configure && make && sudo make install`
16:31:59 <dmlb2000> WooHoo pvfs2-client built...
16:33:10 <dmlb2000> hmmm make install seems to be rebuilding everything...
16:35:06 <robl> dmlb2000: so a 'make kmod_install' will take care of installing kernel module and kernel tools
16:35:18 <dmlb2000> ah okay...
16:35:24 <robl> dmlb2000: you'll need to load the module and start pvfs2-client
16:35:33 <robl> dmlb2000: *then* mounting should work
16:36:00 <dmlb2000> yeah I had the module loaded... fyi rhel5 needs the --disable-kernel-aio on configure...
16:38:16 <dmlb2000> Thanks, that worked great.
16:38:39 <robl> dmlb2000: ok, that's too bad about --disable-kernel-aio, but thanks for letting us know
16:38:45 <robl> dmlb2000: you've got it mounted now?  cool
16:39:31 <dmlb2000> Okay so as a general rule, make && make install work for servers and make kmod && make kmod_install for the clients then?
16:41:10 <robl> dmlb2000: I'd recommend doing both parts for clients.  clients can make use of some headers, libraries, and some utilities.   
16:41:23 <robl> dmlb2000: there's a --disable-server if you really want a client-only installation
16:41:28 <dmlb2000> Also, has anyone thought about providing a spec file or some debian/* files for making packages?
16:41:31 <dmlb2000> Okay thanks...
16:42:07 <dmlb2000> and if not would you be willing to accept some? ;)
16:45:01 <robl> dmlb2000: i think we have an (old) spec file?  but if you've got one you like, send it along (or even better, post to the mailing list)
16:45:20 <dmlb2000> k, I'll do that :)
16:45:43 <robl> as for debian/* files, it's my understanding that they sometimes get in the way if they are in upstream. better to have some debian person maintain it full-time
16:45:45 <robl> but that
16:45:55 <robl> that's just my (limited) understanding
16:46:29 <dmlb2000> Yeah, I've seen both with debian files...
16:46:46 <dmlb2000> only a handful do but most don't
16:48:59 <dmlb2000> robl: some of the pages talk about running pvfs2-client as a non-root but can you run pvfs2-server that way too?
16:59:04 <dmlb2000> hmmm, probably should go to the ml... these next few questions I have are a little much for irc
17:10:35 *** |icebone| has quit ("KVIrc 3.2.5 Anomalies http://www.kvirc.net/")
17:17:24 <Kyle_AmesLab> dmlb2000:  there have been tests on large bluegene installations with >8k clients, not sure if anyone here runs >128 servers, I suppose that could be the case if one were to use localdisks on compute nodes as servers too..
17:23:59 <dmlb2000> Kyle_AmesLab: yeah basically what we want to try is wide striping runs attempting to use one gigantic filesystems with equal numbers of servers and clients...
17:25:07 * dmlb2000 points at the SciDAC PDSI we page.
17:25:33 <Kyle_AmesLab> dmlb2000: I've messed around with client:server ratios of close to 1 for a while now... its frustrating business trying to squeeze every ounce of performance out of systems at that level.
17:26:09 <dmlb2000> agreed. ;)
17:26:48 <Kyle_AmesLab> dmlb2000: whats the interconnect look like on your system?
17:26:58 <dmlb2000> quadrics
17:27:25 <Kyle_AmesLab> dmlb2000: nice, is that 10g?
17:27:41 <dmlb2000> they suply a quadrics mpi with their hardware, but I haven't tried to compile pvfs2 against it...
17:27:51 <dmlb2000> hmmm, not sure... let me look
17:29:10 <dmlb2000> http://mscf.emsl.pnl.gov/hardware/config_mpp2.shtml pfft, it just says QsNET II not sure what's the bandwidth there...
17:30:14 <dmlb2000> Yup, I think that's 10g.
17:30:51 <Kyle_AmesLab> cool cool.
17:31:25 <Kyle_AmesLab> dmlb2000: yeah, it seems you might benefit from a shared space, considering the 10:1 ratio of localdisk to shared disk that system has.
17:32:32 <Kyle_AmesLab> dmlb2000: whether or not IA64/pvfs2 play nice is another question, I dont know if I've seen any tests on itanium archs, I'm sure someone has tried it before
17:33:07 <dmlb2000> okay, well there's been a bunch of 'first time for everything' with itaniums here ;)
17:34:04 <dmlb2000> and I've got it working on my desktop ia64 box using rhel4 did a few local tests to make sure they work... 
17:34:13 <dmlb2000> s/rhel4/rhel5/
17:35:14 <Kyle_AmesLab> dmlb2000: lol, true, things should work themselves quickly here, everyone here is quick to fix issues. 
17:36:43 <dmlb2000> yeah I'll be sure to push results and bugs back after the test next time we reboot the box...
17:38:28 <Kyle_AmesLab> :-)
17:39:30 <Kyle_AmesLab> I'm interested to see whatever results you get, we're limited to about 40 clients/10 servers all on really fast hardware and have seen okay results thusfar
18:23:54 <Kyle_AmesLab> slang: hrmm, i sent pw a set of patches, I'm wondering if he's on vactation or something, he hasnt responded for a couple days
19:52:12 <robl> Kyle_AmesLab: hey, when are your start/stop dates at anl?
19:52:28 <Kyle_AmesLab> robl: may29th - aug3rd
19:53:37 <robl> Kyle_AmesLab: are you a grad student?
19:54:09 <Kyle_AmesLab> robl: nope nope
19:54:28 <robl> Kyle_AmesLab: suli?
19:54:37 <Kyle_AmesLab> robl: yup
19:55:59 *** Kyle_AmesLa1 (n=kyle@12-216-6-144.client.mchsi.com) has joined #pvfs2
19:56:04 <Kyle_AmesLa1> ..stupid router
20:13:54 *** Kyle_AmesLab has quit (Read error: 110 (Connection timed out))
21:33:31 * dmlb2000 is sad that pvfs doesn't have DESTDIR in their build system...
22:36:46 <Kyle_AmesLa1> dmlb2000: destdir?
22:50:58 <dmlb2000> yeah standard automake creates it, its a variable that prefixes the prefix ;)
22:51:41 <dmlb2000> so prefix is where the binary should be expected to be run from, DESTDIR is where the binary should be expected to be installed to.
22:51:50 <robl> Kyle_AmesLa1: ah, i vaugely remember that variable being important when you say make an RPM or debian package
22:52:41 <dmlb2000> yeah it is... have a patch to add it just need to get past that stupid configure issue... *sigh* I'm about ready to fix the redhat headers an move on...
22:53:07 <Kyle_AmesLa1> oic
22:56:27 <robl> dmlb2000: man, our configure tests are doing you no favors at all
22:56:40 <dmlb2000> nope... *sigh*
22:57:30 <dmlb2000> I'll have to say that this is a way cleaner system than other parallel filesystems I've messed with...
22:58:11 <robl> dmlb2000: it looks like for your workaround you can set RPM_OPT_FLAGS in the spec file
22:58:26 <dmlb2000> yeah might have to do that...
22:58:35 <robl> and we'll re-write the tests to not rely on -Werror
22:58:44 <robl> (which we've done for some but not all)
22:58:57 <robl> ((man... testing that is going to stink...)
22:59:00 <dmlb2000> but you'll find that most distro's pass the -O2 to gcc by default and its extra work on package managers to remove it
22:59:19 <dmlb2000> another cool thing would be to add a --disable-werror on the configure script... 
22:59:55 <robl> dmlb2000: well, we only use it for certain configure tests where we really do want AC_TRY_COMPILE to die
23:00:11 <dmlb2000> ah okay
23:00:28 <robl> so that's back to the "real" fix: redo the configure tests
23:00:38 <robl> which would also help us on some other platforms too
23:00:43 <dmlb2000> yup...
23:02:08 <robl> so as you may have guessed, we don't get a lot of people packaging up PVFS in an RPM. thanks for making the effort!
23:03:30 <dmlb2000> np, I really like making package managers do the work they are supposed to ;)