Building and running the Hercules IBM mainframe emulator on VMS

Hercules is an open source software implementation in C of the mainframe System/370, ESA/390 architectures and 64-bit z/Architecture. Hercules runs on linux, windows, Mac OS X and various other unix variants.

Hercules release version website

VMS provides a posix environment which allows portable software like Hercules to run with minor modifications. This enviromnent is available on all VMS platforms including the original VAX hardware as well as Alpha, Itanium and beyond. However, support for kernel threads and compiler support for 64 bit integer quantities is only provided on the Alpha and later platforms. Hercules relies extensively on the latter two facilities. It is not impossible to build Hercules on a VAX, particularly on later generation faster VAX machines with lots of memory, however it requires a great deal of hacking and performance is very poor. In comparason, building on the even the earliest Alpha and on subsequent platforms is very straightforward and performance is very usable.

Here is an updated procedure to build Hercules for VMS on Alpha platforms and beyond:

Building Hercules on VMS (version 1.3)

This includes improved Hercules control panel support making it possible to use the command history feature and to edit Hercules commands while entering them and to page back and forth through Hercules console output. It is also possible to include support for 64 bit pointers which allows Hercules to access many multiple GB of memory, if required. Other minor bugfixes include the ability to recognise and report the number of host CPUs when running on multiprocessor systems and better support for running Hercules detached, including redirecting the Hercules command console to another terminal, including DECterms for example.

This procedure can be used to build the most recent production release of Hercules available from here:

Hercules 3.13 source download

It can also be used to build the very latest post release version 3.13 code available for download here:

Latest Hercules source code repository

This includes support for new features such as TCPNJE devices which are not present in the released version 3.13 as well as several bugfixes and minor enhancements.

Here is a slightly earlier procedure which is lacking some bugfixes and device support:

Building Hercules on VMS (version 1.2)

Below is a description of a slightly older method of building Hercules 3.13 on VMS, with 32 bit pointer support only meaning it is not possible to access more than one or two GB of memory from within Hercules:

Building Hercules 3.13 on VMS with 32 bit storage (version 1.1)

A less easy approach to building Hercules 3.11 on VMS is also provided here:

Building Hercules 3.11 on VMS (version 1.0)

(I used to provide this in 'patch' format for Hercules 3.07 here but I think VMS users may prefer to work with a description rather than to have to track down and build yet another package to process the patches).

Running Hercules on VMS

Once hercules.exe has been built, the most flexible way of running it is to set up a foreign command symbol like this:
$ hercules:==$device:[directory]hercules
where device:[directory] specifies the location of hercules.exe. Then you can use commands such as:
$ hercules -b nla0: -f configfile.cnf
to launch Hercules. In this example, -b specifies a replacment Hercules logo file which is displayed on 3270 terminals connected to Hercules before IPL. In VMS, nla0: specifies the null device which returns nothing when read from, so this example specifies that the usual Hercules logo file should not be output.

If it is wished to redirect Hercules output to a file, I suggest using the Hercules console command 'log' rather than using pipes and ">" output redirection.

If you use a non-default sized terminal window to run Hercules, DCL symbols or logical names LINES and COLUMNS should be defined to advice Hercules of the terminal dimensions.[1] For example, you could include:

$ Define Lines 'F$GetDvi("Sys$Output", "Tt_Page")'
$ Define Columns 'F$GetDvi("Sys$Output", "DevBufSiz")'
in your login.com file, after a SET TERMINAL command to set the terminal size.

[1] There is no need to define LINES and COLUMNS logical names in login.com etc if using recent build procedures as user mode logical names are automatically and transparently defined by the code in this case.

Connecting 3270 sessions to Hercules

Multinet TN3270 emulation

If you have Multinet TCP/IP software installed, the Multinet TN3270 emulator can be used to connect to Hercules, for example:
$ telnet 127.0.0.1 /port=3270 /tn3270=force
A newer more capable (and slower) TN3270 emulator may be selected instead by issuing:
$ define MULTINET_TN3270_EMULATOR DPC

or

$ define MULTINET_TN3270_EMULATOR DPC_EXTENDED
before the telnet command. (Add /nocolor to the telnet command to avoid the garish colours which will then result!)

The file map3270.dat is used to specify the mapping between whatever keyboard is in use with VMS (including terminals defined in sys$system:termtable.txt) and the functions of a 3270 terminal keyboard. Here is a sample file which I find useful for working with Hercules on VMS: map3270.dat. This file can be placed in the multinet: directory to replace the default map3270.dat file supplied with multinet. Alternatively, if it is only required for certain users, it can be placed elsewhere and a logical name MAP3270 defined to point to it (with an @ sign in front of the filename) just for the userids that wish to use it.

There may be some difficulties with certain keyboard mappings and also certain screen character mappings when using Multinet 3270 emulation in DPC or DPC_EXTENDED mode. The characters involved tend to be used mostly when dealing with C source: backtick ` tilde ~ square brackets [ ] and curly brackets { }

Difficulties with these keyboard characters can be overcome by adding the following (to the vt100 definition for example) in map3270.dat:

ebcdic_79 = '`';
ebcdic_a1 = '~';
ebcdic_ad = '[';
ebcdic_bd = ']';
ebcdic_c0 = '{';
ebcdic_d0 = '}';
ebcdic_e0 = '\\';

However, this will result in errors such as "ebcdic_79: unknown 3270 key identifier" when the non-extended and non-DPC version of Multinet tn3270 is used unfortunately.

It is also possible to patch multinet:telnet.exe to deal with incorrect output character processing which results in the incorrect display of square brackets [ and ]. For example, the following works with some variants of Multinet 5.5 for Alpha:

$ patch /absolute multinet:telnet.exe
replace /byte 4e6fd=0dd
5b
exit
replace /byte 4e70d=0a8
5d
exit
update

It may be necessary to tweak the patch for other variants.

C3270 TN3270 emulation

It is also possible to hack the C3270 package to enable it to build on VMS, however, results are not entirely satisfactory. It may also be possible to build the X3270 variant but I haven't looked into this to the degree necessary.

VMS specific aspects of running Hercules

File specifications

The location of DASD files, printer files etc in the Hercules configuration file or in commands on the Hercules console should be entered in their VMS format:

node::device:[directory.subdirectory]filename.filetype;version
although in some circumstances, Unix type file specifications such as:
/directory/subdirectory/filename
may also be usable, especially where ODS-5 disks are in use.

Shutting down or interrupting Hercules

It is normally recommended to shut down Hercules cleanly using the quit command on the Hercules console. If Hercules is inadvertently interrupted by Control-Y, give the VMS command CONTINUE to resume running Hercules.

If Hercules somehow ends up in a state where it will not accept commands on the Hercules console, it may be necessary to use Control-Y followed by a STOP command in order to return to DCL.

HTTP server

Using the inbuilt webserver to control Hercules when running on VMS

Sockdev devices

Issues have been reported attempting to use sockdev devices on VMS, including for example errors like this:

HHCSD014E Failed to bind or listen on socket localhost:3505: invalid argument
HHCCF044E Initialization failed for device 001C
Depending on the TCP/IP stack in use, this may be due to the use of a zero value for the backlog in calls to listen() in sockdev.c. If this is experienced, in sockdev.c, try changing this line (in two places):
        || listen (sd, 0) == -1
to
        || listen (sd, 1) == -1
Then recompile sockdev.c and relink Hercules.

Sockdev devices using "Unix sockets" are unlikely to work unless supported by the VMS TCP/IP stack in use.

Facilities not available when running on VMS

I have not included support for some of the less portable aspects of Hercules when running under VMS. These include direct access to SCSI tape devices and low level network interfaces such as CTCI, CTCT and LCS. If required, VMS versions of these facilites could be developed, however there does not seem to be any great demand for them to justify the work involved.

On the other hand facilities such as TCPNJE and VMIUCV & VMTCPIP were originally developed on VMS by me so will have no difficulty operating on VMS.

Support for dynamically loadable modules and device handlers (HDL) is not provided due to the extremely non-portable implelementations in use by Hercules in ltdl.c et al. This means that facilities which were implemented using dynamic loading only without any regard to use with a statically linked Hercules executable are not straightforward to make use of under VMS. One such example is integrated consoles (1052-C and 3215-C), however, it is possible to get them to operate by cheating slightly, if only I could remember how I did it and what the catch was...

Mail _software_@_beyondthepale_._ie_ (without the underscores) for further information or help with any problems building or running Hercules on VMS.


Page updated July 2024