July 18, 2006 - George H. Fisher ---------------------------------------------- The C/Fortran callable version of SDF: ---------------------------------------------- The Fortran and C callable version of the set of SDF functions is written in C. The source code for version-0.67 of the SDF i/o software is contained in 2 files, sdf_subs.c and sdf_subs.h Thus far, the SDF code has been successfully compiled and tested in Linux (x86 and x86_64), SUN-Solaris, Mac OS X, and Windows XP, using many different versions of gcc, as well as the Intel compiler icc, the Portland group compiler pgcc, the SUN compiler, and the Microsoft Visual C/C++ compiler. There is a Makefile; to compile, just type make. In a unix environment, this should create a linkable object, sdf_subs.o, and a library, libsdf.a . It will also compile an executable command-line sdf file browser, sdf_browse , which can be used to perform a cursory examination of the contents of an sdf file. Typing "make all" will also make almost all of the example programs, but this will require that you have a functioning fortran compiler for some of the examples. The default compiler for SDF is gcc. If you want to use a different compiler from gcc, change the COMPILER setting in Makefile. If you are on a Solaris system make sure you use gnu make ('gmake') and not Sun's make, which seems to generate spurious error messages about unexpected ends of lines. Make clean will remove all the objects, libraries, the sdf_browse executable, and the example programs xtestmain, xtestf77, xtest_sdf_edit, xlarge, x3d, x4d, x5d, xtc89, xtc99, xtest_dyn_f95, xtest_transp_f95, xtestrbwb, and xtestt. Make all will compile all the C and Fortran test programs (except for xtc99), as well as the libraries etc. This will require that you have fortran compilers defined. Make sdf.dll will only work in Windows (see below). The test executable xtc99 requires complex variables to be defined in C which is only true for C99 compliant compilers. There is an entry in the Makefile for xtc99, but it is not included in 'make all'. In MS Windows, the make file works fine under the mingw and msys environments, at least using the mingw 3.1.0-1 installation package (MinGW-3.1.0-1.exe) , and version 1.10 of msys (MSYS-1.0.10.exe) . More recent MinGW versions, such as 5.02, also seem to work fine (but are missing the debugger gdb). Mingw and msys are free software that can be downloaded from www.mingw.org . Mingw stands for "minimal gnu for windows" and provides a self-consistent package of the gcc compiler suite and gdb debugger that link against existing Microsoft C libraries that are present in most windows installations. Msys provides a unix-like shell environment from which the mingw compilers and debuggers and make commands can be run. Programs successfully compiled under mingw should work on most windows machines. Mingw only includes the Fortran 77 compiler g77 as part of the compiler suite, but a free fortran 95 compiler, g95, that works with mingw can be downloaded from www.g95.org . One can also now download mingw compatible versions of gfortran, the open source fortran 95 compiler that is part of gcc version 4. There are several versions of compiler flags in the Makefile, either for debugging or for optimization. Pick the version you want and uncomment it. If you pick a version with profiling ( with the -pg option) you might need to add a -lgprof in addition to -lsdf in the appropriate lines of the Makefile if you are running in MinGW. To install the library and include file into the usual spots in /usr/local, on a unix system, become root and type "make install". This will also install an executable sdf file browser program sdf_browse in /usr/local/bin . If you want to change the prefix to some other location, change the PREFIX variable in the Makefile. Make uninstall will remove the sdf_browse executable, the libsdf.a library, and the sdf_subs.h header from the PREFIX locations used in "make install". To link the sdf library into your fortran program barf.f creating executable xbarf, these commands should work (according to these 4 compilers: g77: g77 barf.f -lsdf -o xbarf g95: g95 barf.f -lsdf -o xbarf ifort: ifort barf.f -lsdf -o xbarf gfortran: gfortran barf.f -lsdf -o xbarf If the compiler/loader complains about missing symbols for various sdf functions, try setting -L in the compiler commands above, where is the directory where the library libsdf.a is located. Important Fortran note: The sdf software assumes the Fortran standard for the size of integers passed as arguments, which is sizeof(double)/2. If you change the integer size from sizeof(double)/2 in the integer arguments used in calling sdf functions, you must make a consistent typedef for variable type intf in sdf_subs.h, and you must modify the test for intf size in test_sizes() in sdf_subs.c accordingly. If you don't do this, strange and hard-to-debug things may happen. To use the functions sdf_wb_f77 and sdf_rb_f77, two of the arguments (datapos and nelem) must be 64-bit fortran integers -- integer*8 or integer (kind=8) . To link the sdf library into another C program barf.c to create executable xbarf: gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE barf.c -lsdf -o xbarf The C program barf.c will need to include the line #include in the declarations part of the code. There is a hook in the make file for my_C_prog, which could be customized for putting in your own C program that links with the sdf library. If the compiler can't find the header sdf_subs.h, insert a -I into the compiler command, where is the directory containing sdf_subs.h . Similarly, if there are undefined sdf symbols from the loader, insert a -L into the compiler line, where is the directory containing libsdf.a . USING THE SDF.DLL FILE IN WINDOWS: The MinGW Makefile includes an entry for creating a dynamic link library sdf.dll and definitions file sdf.def if you are using mingw in windows ("make sdf.dll"), in addition to the default compilation and use of the static library libsdf.a . Fortran and C programs compiled under MinGW can successfully link to the dll by just adding the name of the dll file, sdf.dll , to the list of source code files in the compiler command. In general, running the resulting executable will then mean you need to have the sdf.dll file in the same directory as the executable. Typing "make sdf.dll" will also create the zip file described below. For those who don't want or need to install MinGW on their Windows machines, a copy of sdf.dll and sdf.def are included with this distribution in the .zip file, sdf_dll.zip . Just unzip the zip file and it will inflate the sdf.dll and sdf.def files. The sdf.dll and sdf.def files can then be used to link to other programs compiled with other compilers in MS Windows, as described in the next few paragraphs. To link sdf.dll into a non-MinGW compiled program, there are 2 additional steps: (1) create an import library for sdf.dll that will be recognized by the other compilers, and (2) then link the import library with the other source code files that call sdf functions. If you are using the Microsoft Visual C/C++ compiler (msvc), the import library sdf.lib can be created by using the "lib" command in one of the "build" command-line windows: lib /machine:i386 /out:sdf.lib /def:sdf.def This command uses both the sdf.dll file, and the sdf.def definitions file that mingw produces when you type "make sdf.dll". Examples of compiling the import library into other C programs with msvc can be found in the file Makefile_msvc that is included in this distribution. To compile test versions of sdf_browse_msvc.exe and xtestmain_msvc.exe, which use the sdf.dll , bring up one of the Microsoft Platform SDK command-line build windows, get into the sdf source code directory, and type "nmake /f Makefile_msvc" and "nmake /f Makefile_msvc xmaintest_msvc.exe", respectively. These executables should do exactly the same thing as the MinGW compiled programs sdf_browse.exe and xtestmain.exe . If you want to use sdf.dll with the command-line version of the Borland C/C++ compiler (version 5.5, which is available for free), the import library sdf.lib is created like this: implib -a sdf.lib sdf.def sdf.dll Before compiling any programs that use sdf, the include file sdf_subs.h must be edited -- in the Borland compiler, a "long long" variable does not exist, and variable type fpos_t is only 4 bytes long instead of 8 bytes. Near the top of the sdf_subs.h file add this line: #define WIN32 Then, in the typedefs for variable type "pos", substitute __int64 for fpos_t, and in the typedef for "i8", substitute __int64 for long long. The example commands below assume that sdf_subs.h is then placed into the default Include directory for the Borland compiler, (which is commonly C:\Borland\BCC55\Include ). To compile e.g. a version of sdf_browse.exe using the Borland compiler bcc32, the next steps would then be: bcc32 -c sdf_browse.c bcc32 sdf_browse.obj sdf.lib The resulting version of sdf_browse.exe should behave in just the same way as the MinGW compiled version of sdf_browse. EXAMPLE PROGRAMS THAT USE SDF: Examples of the usage of SDF i/o functions can be found in sdf_f77_tests.f for Fortran 77, and main_test_sdf.c, sdf_test_large.c, test_sdf_3d.c, test_sdf_4d.c, test_sdf_5d.c, test_complex_c89.c, test_complex_c99.c, test_sdf_large.c, and sdf_test_edit.c for C. The first Fortran example should compile with Fortran 77 compiler such as g77 or pgf77. Examples of fortran usage with allocatable arrays are given in test_sdf_dynamic.F90 and test_sdf_transpose.F90 . Here, you'll need to use a Fortran 90/95 compiler such as g95, gfortran, or ifort. These use sample sdf files bz.sdf and sdf_example_file.sdf , the latter of which is included in the distribution. The file bz.sdf contains the vertical magnetic field from a snapshot of an anelastic 3-D MHD simulation run by Dave Bercik. This file is not included in the distribution by default, because it is so large (20 MB), but it can be downloaded from http://solarmuri.ssl.berkeley.edu/~fisher/public/software/SDF . There are entries in the Makefile for making the example programs ("xtestmain", xtest_sdf_edit", "xlarge", "x3d", "x4d", "x5d", "xtc89", "xtc99", "xtestf77", "xtest_dyn_f95", and "xtest_transp_f95"), but before using the latter 3 make sure that you set the Fortran compiler names in the Makefile to the compiler you have installed on your system. Note that as of late 12/22/05 large files (> 2GB) appear to be working properly at least for the cases I've considered thus far, at least in linux and windows. As of 6/20/06, large files also appear to be working for OSX on Mac G4 and G5 machines, as well as on SUN-Solaris machines, compiled with either the SUN default compiler or with gcc. The program "xlarge" tests for the ability to write large files. Make sure you have enough disk space before you try to run this! Make sure you are not using a FAT32 partition! On a modern laptop, this may take about 20 minutes to run to completion. Again, note that in Windows, you must be using an NTFS partition (and not a FAT32 partition) for this to work. I have only confirmed the MinGW compiled version as working correctly with large files. I suspect msvc compiled programs will also work correctly, and am cautiously optimistic but less certain about Borland compiled programs. --------------------------------------------------- The IDL version of SDF: --------------------------------------------------- Put all the files in the IDL folder somewhere in your IDL path, and that should be all that is necessary to install the IDL version of SDF. IDL functions sdf_read, sdf_write, sdf_query, sdf_sizes, sdf_insert, sdf_delete, sdf_insert mirror the C/Fortran callable functions, and are "self-documenting", at least as far as the calling syntax goes; typing in the name of the procedure should spit out the correct calling syntax. Try exploring the contents of the two example sdf files included in the distribution (sdf_example_file.sdf and bz.sdf) with sdf_query, sdf_read_all, etc. The IDL versions of sdf_insert, sdf_replace, and sdf_delete are pretty slow when applied to datasets near the front of very large ( > 2GB) files. The C/Fortran callable versions seem to run faster.