Porting Open Source applications on MPE/iX A case Study of Samba-3.0.22 Author: Vidya Sagar Version 1.
Table of Contents Revision History..............................................................................................................................................................iii Executive Summary ....................................................................................................................................................... 1 Introduction ...................................................................................................................................
B. C. D. E. F. G. H. I. Installation Notes (Autopat and Patch/iX) ...................................................................................................................31 Script to install GNU gcc compiler and utility (INSTALL.gcc) ...............................................................................31 Script to install POSIX wrappers (INSTALL.px_wrappers) ....................................................................................32 Script to install BSD library (INSTALL.
Executive Summary This paper is intended to illustrate the details of porting one of the most popular and useful open source applications, Samba version 3.0 22, from the internet and interoperability domain. The paper should provide sufficient information to 1) Refresh a new version of Samba on MPE/iX, 2) Apply future patches released by Samba organization (www.samba.
. All the links and bookmarks in this document are blue underlined. 6. All the examples and command demonstration have used 4 way series 969-400 system installed with MPE/iX C.65.00 FOS, POSIX version A.50.02, Samba-3.0.22, and gcc version 3.2. 1 Background on MPE/iX, POSIX compliance, and GNU Tools In this section we will review some general concepts regarding MPE/iX, its implementation of POSIX.1 and POSIX.2 standards and the GNU (or “Open Source”) tools.
However, MPE interfaces such as intrinsic(s) and CI commands escape to the POSIX name server when the file name begins with the “.” or “/” escape characters. File Access and Security: In MPE/iX, POSIX 1003.1 file permission bits have not been implemented as a separate access control mechanism. Instead, POSIX 1003.1 functions support the file permission bits via the MPE ACD mechanism. ACDs themselves have been enhanced to enable the ACD mechanism to operate as a POSIX 1003.
your current working directory. If makefile does not exist, then make searches for a file named Makefile. If you don't want to use one of the default names, other files can be used with the “-f command-line” option. The convention used to identify makefiles not named makefile or Makefile is to use the .mk suffix (for example, foo.mk). autoconf: Autoconf is a tool for producing shell scripts that automatically configure software source code packages to adapt to many kinds of Posix-like systems.
MPE/iX header files and libraries It is a good idea to be familiar with what header files and libraries are available on MPE/iX which will be used by the portable applications. In this section we will discuss the header and libraries which are a part of FOS as well as other headers and libraries like libbsd and POSIX wrappers required for porting applications. Please refer to the Build Machine configuration section which discusses how to install libbsd and POSIX wrappers. 1.A.
1.A.2 MPE/iX Libraries Normally, an application required to be linked with many libraries. Some application requires library only at compile time and some at runtime as well. Say for example Samba requires the POSIX wrappers at runtime also; hence it becomes necessary to include the runtime library into the package. MPE/iX system libraries are available in two types NMRL archive library and NMXL shared library. NMRL and NMXL are available in MPE namespace in as NL.PUB.SYS and XL.PUB.
int setgid(gid_t); int setuid(uid_t); The non-simulated version of setuid() must be invoked in privileged mode. The simulated version takes care of this as well as one change directory bug listed below. setgid() also required in privileged mode ( Invoke GETPRIVMODE()) but does not do anything as MPE/iX does not support direct gid switching. Please see notes on supplementary groups discussed later.
9 The maximum size of any single socket I/O request on MPE/iX is 30,000 bytes. Many applications will natively attempt do to I/O in chunks of 65,536 bytes (or larger). These I/O calls will fail on MPE, and you will need to modify the application to do socket I/O in smaller chunks 9 MPE/iX bind() should be invoked in privileged mode to bind privileged (<1024) ports. Also, bind() binds only to wildcard interfaces. 9 MPE/iX cannot connect to datagram sockets.
As a base for your porting efforts we strongly recommend that your build machine be running MPE/iX 6.5, 7.0 or 7.5 preferably with the POSIX shell version A.50.02. The POSIX shell and libraries are part of the MPE/iX Fundamental Operating System (FOS) and are assumed to be available on any MPE/iX machine. Installation of MPE/iX OS FOS and Subsystems is beyond the scope of this paper.
2.A.2 BSD libraries and headers As discussed in the MPE/iX header and libraries section the build machine should have the libbsd package installed. Install the BSD libraries and headers available at http://jazz.external.hp.com/src/bsd/libbsd.html?jumpid=reg_R1002_USEN. The steps are: 1) Download the libbsd package from the link specified above to your windows machine. 2) FTP that to the target machine inside the /tmp directory.
DISC SPACE: 11327520(SECTORS) PASSWORD: ** CPU TIME : 725144(SECONDS) LOC ATTR: $00000000 CONNECT TIME: 992678(MINUTES) SECURITY--READ : ANY DISC LIMIT: UNLIMITED WRITE : AC CPU LIMIT : UNLIMITED APPEND : AC CONNECT LIMIT: UNLIMITED LOCK : ANY MAX PRI : 150 EXECUTE : ANY GRP UFID : $05670001 $0981A6C5 $000045CE $130028B8 $1B747DCB USER UFID: $00000000 $00000000 $00000000 $00000000 $00000000 CAP: SM,AM,AL,GL,DI,OP,CV,UV,LG,PS,NA,NM,CS,ND,SF,BA,IA,PM,MR,DS,PH The account structure for Samba can then be create
Normally POSIX environment are controlled by various shell variable, which you can define in the .profile file in home directory or home group. On entering into the POSIX shell, this file is read and sets the variable if specified. Edit that file to include the following entries: #create some aliases alias ll="ls -l" alias lsf="ls -F" alias mpe="callci ci,2" alias scan=”find .
10. Determine unresolved symbols. 11. Resolve unresolved symbols and rebuild and install. 12. Test all the features you think should be supported in step 1. 13. Document the unsupported feature. 14. Finally bundle the deliverables, documentation and release. A fresh port does not involve step 4 to apply a patch. Rather the porter modifies the necessary configuration files (configure.in, Makefile.in etc) manually wherever required which is here done by the POSIX command patch.
Assuming you have the samba source code (e.g smb3022.tar.gz) downloaded into your Windows workstation, following example shows how to ship (PUT) them to your build machine using FTP: Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\sagarvi>cd c:\ C:\>cd downloads C:\Downloads>dir Volume in drive C is E-Client Volume Serial Number is 88B7-64DC Directory of C:\Downloads 05/22/2007 05:15 PM 18,234,944 smb3022.tar.
Now you should have a file named samba-3.0.22.tar. Typical uncompressed tarballs are created with relative pathnames and will eventually restore files relative to the current working directory. It is usually a good idea to know in advance what is the content of tarball and that can be achieved by issuing tar –tvf which reveals the directory structure to be created and files it contains. Next is to extract the contents of tarball. Shell/iX> tar –xvf samba-3.0.22.tar The directory samba-3.0.
shell/iX> pwd /SAMBA/SMB3022/src shell/iX> /usr/local/bin/diff -ruN samba-3.0.22/source samba-3.0.22-mpe/source >./diff.txt The diff -r option says to recursively compare two directory trees. The -u option says to generate output in the "unified" format. The -N option says to generate entries to create files that exist only in the second directory tree. The generated diff.txt file looks something like this: diff -ruN samba-3.0.22/source/Makefile.in samba-3.0.22-mpe/source/Makefile.in --- samba-3.0.
*mpeix*) export MPEAUTOCONF=1 CPPFLAGS="$CPPFLAGS -Dmpeix -D_POSIX_SOURCE -D_SOCKET_SOURCE -D_INCLUDE_MPEXL_SOURCE -I/SYSLOG/PUB -I/usr/contrib/include" LDFLAGS="$LDFLAGS -L/POSIXC60/lib -L/SYSLOG/PUB -Xlinker -WL,xl='/$HPACC OUNT/$HPGROUP/LIBCPXL'" LIBS="$LIBS -lposix60 -lunix -lsyslog -lsocket -lsvipc -lcurses" AC_DEFINE(mpeix,1,[Target is MPE/iX]) AC_DEFINE(_POSIX_SOURCE) AC_DEFINE(_SOCKET_SOURCE,1,[Socket source]) AC_DEFINE(_INCLUDE_MPEXL_SOURCE,1,[Include MPEXL source]) AC_DEFINE(USE_SETREUID,1) AC_DEF
shell/iX> pwd /SAMBA/SMB3022/src/samba-3.0.22-mpe/source shell/iX> ./autogen.sh ./autogen.sh: running script/mkversion.sh ./script/mkversion.sh: 'include/version.h' created for Samba("3.0.22") ./autogen.sh: running autoheader autoheader: `include/config.h.in' is created ./autogen.sh: running autoconf Now run ./configure.mpe and then make proto and finally make. The configure.mpe looks like: shell/iX> cat ./configure.mpe #!/bin/sh .
It takes almost 5~6 hours on HP e3000 969-400 machine to complete the full configuration for Samba. Upon successful completion of configure, you will find some new Makefile or other modified files for use during build. Analyze the file config.log for any errors during configuration. If there is any error, modify configure.in to address the error and proceed again by running autogen.sh.
#else #define MAX_SOCKET_IO_LEN SSIZE_MAX #endif /* File: ./lib/system.c */ /******************************************************************* A sendto wrapper that will deal with EINTR.
Please note that many header files have been created. After running configure and building prototypes its time now to build your application. 3.8 Build (make and make install) Some portable applications have only one Makefile at the root of source tree and some have many Makefiles each in a subdirectory which contain rules to build code in that particular subdirectory. In the latter case, Makefile(s) call each other in nested fashion.
If you see both the “linking libmsrpc” and “nnn OBJECT FILES HAVE BEEN ADDED” then the Samba application has been built successfully and is ready to be installed. The moment make encounters any error it stops there with error message on the STDOUT. If you see any error while building, just look into the corresponding file and do the necessary modification. Refer the section common problems and their resolutions for more information.
3.9 Setting proper Capabilities and ownership As we discussed earlier some binaries (smbd, nmbd & swat installed in /$HPACCOUNT/$HPGROUP/sbin) require that they be run as super user that is UID should be set to 0. In MPE/iX that is achieved by allowing the code to run into privileged mode by invoking MPE/iX intrinsic GETPRIVMODE(). In order to GETPRIVEMODE, the binary must be linked with the capability PM and must be moved into the MPE file system name space (that is /ACCOUNT/GROUP/PROGRAM or PROGRAM.
POSIX SHARED DATA TEXT SIZE DATA SIZE VERSION : : : : : YES YES 000F8C10 0003B000 85082112 MPE/iX: :linkedit "listprog smbd.smb3022.samba" HP Link Editor/iX (HP30315A.06.17) Copyright Hewlett-Packard Co 1986 LinkEd> listprog smbd.smb3022.samba PROGRAM XL LIST CAPABILITIES NMHEAP SIZE NMSTACK SIZE ENTRY NAME UNSAT NAME PRIORITY MAX PRIORITY POSIX SHARED DATA TEXT SIZE DATA SIZE VERSION : : : : : : : : : : : : : : SMBD.SMB3022.
Where “theprogram” is the newly linked program, “notfound” is any :FILE equation name that has not been defined, and “tempfile” is a temporary file where any :RUN error messages will be written. If the program contains unresolved external references, the :RUN command will fail, and the list of unresolved symbols will be written to the temporary file “tempfile”. But if the program does not contain any unresolved symbols, the :RUN will still fail because the STDIN :FILE equation does not exist.
#ifdef mpeix ldap_unbind_s(ldap_state->ldap_struct); #else ldap_unbind_ext(ldap_state->ldap_struct, NULL, NULL); #endif If you encounter unresolved symbols which do not seem to be covered by any of the above mentioned class, either refer common runtime problems for other suggestions. Once all the unresolved routines are resolved rebuild the application as mentioned in step 8 and 9. You now should have an application that is built, installed on the build machine and ready to test. 3.
3.15 Performance Notes You may encounter noticeable performance degradation on running some ported application on MPE/iX and some may give good performance also. One reason behind performance degradation I believe might be due to some extra overhead introduced as a result of implementation/simulation of POSIX APIs on MPE/iX. Another is that MPE/iX Process creation is relatively more expensive than on UNIX systems.
But what if it’s not possible to compile with the missing include file? Check to see if the file is provided via the Porting Wrappers (/POSIXC60/include or Appendix H) or in /usr/contrib/include, and if it is, specify either I/POSIXC60/include or -I/usr/contrib/include (but preferably not both) in the CPPFLAGS list of precompiler options.
If the compile error is related to something that an autoconfigure script checked for, verify that the script came up with the correct answer and that the source code actually relies on this answer to determine what to do. We did not encounter this error, hopefully you won’t either. Many socket applications use PF_INET and other PF_xxx macros instead of AF_INET and corresponding macros on MPE. The simplest way to deal with this is to modify your application’s source code to do: #define PF_INET AF_INET 4.
readily apparent from the stack traces of the afflicted process(es). Installing the latest Streams/iX patch for your MPE release is always a good idea before you start your porting activities. Hard Links vs. Symbolic Links The MPE HFS file system does not support hard links. It is frequently acceptable to modify applications to create symbolic links instead. SVIPC Resources Note that MPE does not release SVIPC semaphore or shared memory resources when processes normally or abnormally terminate.
[8] Samba-3.0.22 build document (http://jazz.external.hp.com/src/samba/BuildingSamba-3.0.22.htm) 6 Appendix A. Packaging and Patch Notes for Samba The following information references systems inside HP for use by MPE Lab persons when building a patch for distribution to HP customers; it is included here for completeness. The script to package Samba is available at http://knowmpe.cup.hp.com/krtdocs/Network/ARPAInternetServices/Samba/makedist.txt.
fi else echo Download all the files "file[1-9].bin" or all.bin and try again exit 0 fi fi exit_on_error echo Installing...Please wait... echo Installing...Please wait... compress -d GNU.Z exit_on_error frombyte -b GNU GNU1 exit_on_error callci 'file GNU1=/tmp/GNU1;dev=disc' callci 'restore *GNU1;/;tree;create;show' exit_on_error if [ -f /usr/local/INSTALL.hp3000 ]; then /usr/local/INSTALL.hp3000 exit_on_error echo "***********Installation complete "":)*************" else echo /usr/local/INSTALL.
fi E. Script to install BSD library (INSTALL.libbsd) #!/bin/sh ############################################################### # File: INSTALL.libbsd # Author: Vidya Sagar (vidya.sagar2@hp.com) # Purpose: To install the BSD lib (libbsd.a) into /usr/lib # Assumption: The file libbsd.tar.tar has been downloaded from JAZZ and resides in/tmp ############################################################### #Proceed only if system has /usr/lib dir if [ -d /usr/lib ]; then cd /tmp if [ -f libbsd.tar.
/usr/include/bsd/dirent.h /usr/include/bsd/errno.h /usr/include/bsd/fcntl.h /usr/include/bsd/ftw.h /usr/include/bsd/glob.h /usr/include/bsd/mnttab.h /usr/include/bsd/mpe.h /usr/include/bsd/mpool.h /usr/include/bsd/ndbm.h /usr/include/bsd/nl_types.h /usr/include/bsd/nlist.h /usr/include/bsd/paths.h /usr/include/bsd/pwd.h /usr/include/bsd/resolv.h /usr/include/bsd/sgtty.h /usr/include/bsd/signal.h /usr/include/bsd/stdio.h /usr/include/bsd/stdlib.h /usr/include/bsd/string.h /usr/include/bsd/strings.
/usr/include/bsd/protocols/talkd.h /usr/include/bsd/rpc/rpc.h /usr/include/bsd/sys/cdefs.h /usr/include/bsd/sys/dir.h /usr/include/bsd/sys/errno.h /usr/include/bsd/sys/file.h /usr/include/bsd/sys/ioctl.h /usr/include/bsd/sys/mman.h /usr/include/bsd/sys/param.h /usr/include/bsd/sys/resource.h /usr/include/bsd/sys/signal.h /usr/include/bsd/sys/socket.h /usr/include/bsd/sys/stat.h /usr/include/bsd/sys/statfs.h /usr/include/bsd/sys/stdsyms.h /usr/include/bsd/sys/syslog.h /usr/include/bsd/sys/time.
/POSIXC60/include/stringpac.h /POSIXC60/include/syslog.h /POSIXC60/include/tcmpe.h /POSIXC60/include/termtype.h /POSIXC60/include/turboimage.h /POSIXC60/include/ulimit.h /POSIXC60/include/unistd.h /POSIXC60/include/util.h /POSIXC60/include/sys/mman.h /POSIXC60/include/sys/param.h /POSIXC60/include/sys/socket.h /POSIXC60/include/sys/stat.h /POSIXC60/include/sys/stdsyms.h /POSIXC60/include/sys/time.h /POSIXC60/include/sys/uio.h /POSIXC60/include/sys/utsname.h /POSIXC60/include/sys/wait.h I.
37
