Building Hercules 3.11 on VMS

This document describes a method for building version 3.11 of the Hercules IBM
mainframe emulator on VMS.

Some of the less portable Hercules features are not implemented such as SCSI
tape support, TUN/TAP based networking, FBA block device support and HDL device
support. Hercules devices which are implemented using HDL only will not work.
These features could probably be included given some porting effort, however
there does not seem to be sufficient requirement for them at present to justify
the effort required.

Compilation is done with the default (32 bit) pointer size as there are issues
with the argv pointer size, arguments to setvbuf() and several problems with
getopt() when compilation is attempted with 64 bit pointers.  This suggests
there are likely to be problems trying to configure Hercules with main storage
or expanded storage in the gigabytes range, however, I have not explored this.

A hack is performed to get the softfloat package to compile with least effort.
This means that binary floating point operations may not work correctly if
more than one CPU is configured in Hercules on VMS.

This procedure has been developed and tested on OpenVMS Alpha V8.3 with
HP C V7.1-015 and Multinet V5.2 Rev A.  It should work on other comparable
versions of VMS / C / Multinet / TCPIP Services on Alpha or Itanium without
modifications or with minor tweaks.  Let me know if you experience any problems
and I will try to fix them.

You will need:

 - A VMS version of UNZIP (available from several locations including HP
 - The Hercules 3.11 source hercules-3.11.zip
 - Miscellaneous support files for running Hercules on VMS hercules-vms-support.zip
 - The source for ZLIB (if ZLIB support is required) (version 1.2.3 is suitable)
 - The source for BZIP2 (if BZIP2 support is required) (version 1.0.2 is suitable)

Firstly, unzip the Hercules 3.11 source zipfile in a suitable location on
your VMS system.  Use the -a option on unzip to get the file attributes right.
Secondly, unzip hercules-vms-support.zip in the same location.  This will
result in the creation of a [.vms] subdirectory to contain some of VMS specific
support files.

If you want to compile Hercules with BZIP2 and ZLIB support, the bzlib.h,
zconf.h and zlib.h header files and libbz2.olb and libz.olb object libraries
built from the source should also be placed in the [.vms] subdirectory.  I am
not going to go into the detail of building these packages here as there are
too many variables involved.  If you run into any difficulties with them, let
me know and I will help sort them out.  If BZIP2 and ZLIB support is not
required, comment out or remove the 5 lines referencing ZLIB, LIBZ, BZLIB and
BZIP2 in [.vms]config.h and the 2 lines referencing libz and libbz2 in
hercules.opt.

Eight of the source files supplied with Hercules should be modified as described
below. 

It is necessary to modify hdl.h to fix a bug which causes a compilation error
when compiling without HDL (which is the path of least resistance on VMS). It
is necessary to modify hmacros.h to avoid very poor performance and DBG error
messages on the Hercules console due to calling select() with a non-socket file
descriptor.  The other changes are not essential but recommended, otherwise
different parts of Hercules may fail to work.

If panel.c is not modified, the control panel does not accept keyboard input
and Hercules can only be controlled using it's web interface (which is tricky
to configure correctly but can be done.  Ask me for details if you want it.)

If cardrdr.c is not modified, card readers do not work.  If hetlib.c is not
modified, HET tapes do not work.  If hscutl.c is not modified, 2703
communication adaptors may cause Hercules to hang.

If hostopts.h is not modified, there will be many warnings that default host
options are being used but few consequences.  The changes to hdlmain.c avoids 2
undefined symbols at link time due to lack of TUN/TAP support.  However, there
do not appear to be any ill effects if this issue is ignored.

cardrdr.c

    Locate 'dev->fh = fdopen(dev->fd, "rb");'.  Replace with these 17 lines:

#ifdef __VMS
    dev->fh = fdopen(dev->fd, "r");
#else /* __VMS */
    dev->fh = fdopen(dev->fd, "rb");
#endif /* __VMS */
    if (dev->fh == NULL)
    {
        /* Something went wrong with fdopen */

        logmsg (_("HHCRD020E Error calling fdopen on file %s: %s\n"),
                    dev->filename, strerror(errno));

        /* Set unit check with equipment check */
        dev->sense[0] = SENSE_EC;
        *unitstat = CSW_CE | CSW_DE | CSW_UC;
        return -1;
    }


hdl.h

Locate 'typedef struct _HDLINS'.  Move this 8 line structure up above the
line containing : '#if !defined(OPTION_DYNAMIC_LOAD)'


hdlmain.c

Locate 'HDL_DEVICE(CTCI, ctci_device_hndinfo    );'

Comment out or remove this line.

Locate 'HDL_DEVICE(LCS,  lcs_device_hndinfo     );'

Comment out or remove this line.


hetlib.c:

Locate 'omode = "r+b";'.  Replace with these 5 lines:

#ifdef __VMS
    omode = "r+";
#else /* __VMS */
    omode = "r+b";
#endif /* __VMS */

Locate 'omode = "rb";'.  Replace with these 5 lines:

#ifdef __VMS
        omode = "r";
#else /* __VMS */
        omode = "rb";
#endif /* __VMS */


hmacros.h

Locate '#define  close_pipe(f)        closesocket(f)'.

After that line, insert these 5 lines:

#elif defined( __VMS )
  #define  create_pipe(a)       socketpair(AF_INET,SOCK_STREAM,IPPROTO_IP,a)
  #define  read_pipe(f,b,n)     recv(f,b,n,0)
  #define  write_pipe(f,b,n)    send(f,b,(int)n,0)
  #define  close_pipe(f)        close(f)


hostopts.h

Locate '/* Hard-coded OTHER (DEFAULT) host-specific features and options...  */'
Above this, insert the following 30 lines

/*-------------------------------------------------------------------*/
/* VMS host-specific features and options...                         */
/*-------------------------------------------------------------------*/
#elif defined(__VMS)                          /* VMS options         */
/* Similar to OTHER with minor differences                           */

#define DLL_IMPORT   extern             /* (a safe default)          */
#define DLL_EXPORT

#undef  TUNTAP_IFF_RUNNING_NEEDED       /* (NO tuntap support)       */
#undef  OPTION_SCSI_TAPE                /* (NO SCSI tape support)    */
#undef  OPTION_SCSI_ERASE_TAPE          /* (NOT supported)           */
#undef  OPTION_SCSI_ERASE_GAP           /* (NOT supported)           */
#undef  OPTION_FBA_BLKDEVICE            /* (no FBA BLKDEVICE support)*/

#define MAX_DEVICE_THREADS          0   /* (0 == unlimited)          */
#undef  MIXEDCASE_FILENAMES_ARE_UNIQUE  /* ("Foo" same as "fOo"!!)   */

#define DEFAULT_HERCPRIO    0
#define DEFAULT_TOD_PRIO  -20
#define DEFAULT_CPU_PRIO   15
#define DEFAULT_DEV_PRIO    8

#if defined( HAVE_FORK )
  #define HOW_TO_IMPLEMENT_SH_COMMAND     USE_FORK_API_FOR_SH_COMMAND
#else
  #define HOW_TO_IMPLEMENT_SH_COMMAND     USE_ANSI_SYSTEM_API_FOR_SH_COMMAND
#endif
#define SET_CONSOLE_CURSOR_SHAPE_METHOD   CURSOR_SHAPE_NOT_SUPPORTED
#undef  OPTION_EXTCURS                  /* Normal cursor handling    */


hscutl.c

locate 'int flags = fcntl( sfd, F_GETFL );'.  Above it, insert these 10 lines:

#ifdef __VMS

    u_long non_blocking_option = !blocking_mode;

    if ( ioctl( sfd, FIONBIO, &non_blocking_option) != -1 )
        return 0;

    return -1;

#else /* __VMS */

locate 'return fcntl( sfd, F_SETFL, flags );'.  Below it, insert this line:

#endif /* __VMS */


panel.c

Locate '#define  DISPLAY_INSTRUCTION_OPERANDS'.  After it, insert these 3 lines:

#ifdef __VMS
#include <descrip.h>
#endif // __VMS

Locate '#ifndef _MSVC_'. Replace it with this line:

#if !defined( _MSVC_ ) && !defined( __VMS )

Locate '#endif // _MSVC_'. Replace it with these 6 lines:

#endif // !_MSVC_ && !__VMS
#ifdef __VMS
  int     efn, status, lib$signal(), lib$get_ef(), sys$assign(), vms_getch();
  short   channel;
  $DESCRIPTOR(input, "SYS$COMMAND:");
#endif // __VMS

Locate '/* Notify logger_thread we're in control */'.
Above it insert these 13 lines:

#ifdef __VMS

    /* Reserve an event flag for calling sys$qiow in vms_getch() */

    status = lib$get_ef(&efn);
    if (!(status & 1)) lib$signal(status);

    /* Assign a channel to sys$command for vms_getch() */

    status = sys$assign(&input, &channel, 0, 0);
    if (!(status & 1)) lib$signal(status);

#endif // __VMS

Locate '#else // !defined( _MSVC_ )'. Replace it with these 16 lines:

#elif defined( __VMS )
        /* Wait for keyboard input */
#define WAIT_FOR_KEYBOARD_INPUT_SLEEP_MILLISECS  (20)
        for (i=sysblk.panrate/WAIT_FOR_KEYBOARD_INPUT_SLEEP_MILLISECS;
            i && !(kbbuf[0] = vms_getch(efn, channel)); i--)
            usleep(WAIT_FOR_KEYBOARD_INPUT_SLEEP_MILLISECS * 1000);

        ADJ_SCREEN_SIZE();

        /* If keyboard input has [finally] arrived, then process it */
        if ( kbbuf[0] )
        {
            kbbuf[kblen=1] = '\0';
            translate_keystroke( kbbuf, &kblen );

#else // !defined( _MSVC_ ) && !defined( __VMS )

Locate '#endif // defined( _MSVC_ )'. Replace it with this line:

#endif // defined( _MSVC_ ) || defined( __VMS )

Locate '/* Process the command when the ENTER key is pressed */'.
Above it, insert these two lines:

                /* Translate CR into LF for unix type processing */
                if (kbbuf[i] == '\r') kbbuf[i] = '\n';

That's it.  All source modifications are now done and compilation can begin.

Set default to the [.vms] subdirectory.  Issue this command:

@[-]compile vms

Set default to the [.decnumber] subdirectory.  Issue this command:

@compiledecnumber

Set default to the [.softfloat] subdirectory.  Issue this command:

@compilesoftfloat

Set default to the main source directory.  Issue this command:

@compilehercules

Expect three MAYLOSEDATA2 warnings when compiling cpu.c and another three when
compiling sie.c.  There may also be some informational messages.  If all went
well, all the hercules source should now be compiled.  If so, issue this
command:

@linkhercules

and HERCULES.EXE should be generated.  This can be run with command line
arguments by setting up a foreign command symbol or a simple RUN HERCULES will
run it with no command line arguments and it will look for a configuration file
called hercules.cnf in the current default directory.

If you also wish to build the Hercules support utilites such as dasdinit and
hetinit, issue these commands:

@compileothers
@linkothers

Note that there may be a issue with dasdls.c which may prevent it from
compiling because of a BADCONDIT error.  If this happens, locate
': printf(runflgs' and change it to ': (void) printf(runflgs' before trying
again.  Other than that, there should be no warnings or other messages.

If building without LIBZ and BZIP2 support, comment out or remove the
references to libz and libbz2 from cckd.opt, dasd.opt and het.opt before
linking.

Please note that I have not tested all the support utilities for correct
operation but if any problems are found, I am happy to look into them.

Mail _software_@_beyondthepale_._ie_ (without the underscores) for further
information or help with any problems building or running Hercules on VMS.