Best practices - IBM€¦ · Setting up firewall rules for IBM Platform LSF Page 6 of 17 If you set LSF_ENABLE_EGO=Y in lsf.conf to enable Platform EGO, you must open the following

Issued: September 2014

IBM® Platform LSF ®

Best practices Setting up firewall rules for IBM

Platform LSF

Yuxing Ren

LSF Development

Systems & Technology Group

Setting up firewall rules for IBM Platform LSF Page 2 of 17

Setting up firewall rules for IBM Platform LSF ............................................... 1

Executive Summary ............................................................................................. 3

Introduction .......................................................................................................... 4

Setting up LSF firewall rules in Linux .............................................................. 5

Firewall rules for outgoing packets ............................................................. 5

Firewall rules for incoming packets ............................................................ 5

In a single-cluster environment ............................................................. 5

Open specific ports on every host.................................................... 5

Special requirements for LSF features on every host .................... 6

A simple firewall rule example ........................................................ 6

In a Platform MultiCluster environment .............................................. 7

Setting up LSF firewall rules in Windows........................................................ 7

Opening ports in LSF for Windows ............................................................ 8

Set up LSF firewall rules by using Windows firewall with advanced

security ............................................................................................................ 8

Example Windows firewall configuration ........................................... 9

Best practices ....................................................................................................... 13

Conclusion .......................................................................................................... 14

Further reading................................................................................................... 15

Contributors .................................................................................................. 15

Notices ................................................................................................................. 16

Trademarks ................................................................................................... 17

Contacting IBM ............................................................................................ 17


Executive Summary IBM Platform LSF is a workload management platform and job scheduler for distributed

HPC environments. It is based on networked OS and needs communication between

different hosts through a network. Since firewall services can block any unsolicited

network traffic, setting firewall rules for LSF is required when a firewall service is

running.

This document presents guidelines for how to set up firewall rules for LSF in both Linux

and Windows.


Introduction

This document serves as a best practice guide for how to set up firewall rules for LSF.

This document describes the following:

Setting up LSF firewall rules in Linux

Firewall rules for outgoing packets

Firewall rules for incoming packets

In a single-cluster environment

Open specific ports on every host

Special requirements for LSF features on every host

An simple firewall rule example

In a Platform MultiCluster environment

Setting up LSF firewall rules in Windows

Opening ports in LSF for Windows

Set up LSF firewall rules by using Windows firewall with advanced security

Example Windows firewall configuration


Setting up LSF firewall rules in Linux

Firewall rules for outgoing packets Do not block outgoing packets sent from the machine itself. You should allow all LSF

binaries to send out packets and open all ports that control outgoing packets.

Firewall rules for incoming packets Incoming packets destined for the host computer can be blocked by the firewall. Most

LSF binaries do not need the firewall to open certain ports that control incoming packets,

but bsub –I, lsrun and lsgrun may use specific ports to receive incoming packets from

other hosts.

Note: Some LSF commands, such as bsub –I, lsrun, lsgrun, or bpeek require

incoming packets from LSF on every host. See “Special requirements for LSF features on

every host” below for details.

The following steps are based on iptables in Linux.

In a single-cluster environment

If you use a firewall like iptables to block packets between hosts in the same cluster,

follow the principles below.

Open specific ports on every host

Open the ports shown in Table 1.

Table 1. LSF ports

Port Configuration file Protocol

LSF_LIM_PORT lsf.conf TCP+UDP

LSB_MBD_PORT lsf.conf TCP

LSF_RES_PORT lsf.conf TCP

LSB_SBD_PORT lsf.conf TCP

LSB_QUERY_PORT lsf.conf TCP


If you set LSF_ENABLE_EGO=Y in lsf.conf to enable Platform EGO, you must open

the following EGO-related ports shown in Table 2.

Table 2. EGO-related ports


EGO_KD_PORT ego.conf TCP

EGO_PEM_PORT ego.conf TCP

ESC_PORT egosc_conf.xml TCP

WSG_PORT wsg.conf TCP

53 (the default port of

named)

named.conf TCP+UDP

Special requirements for LSF features on every host

For the busb -I, lsrun and lsgrun commands, if you set the environment variable

LSF_NIOS_PORT_RANGE, you must open a range of ports with TCP protocol whose

range is equal to LSF_NIOS_PORT_RANGE. Because these features use random ports,

you should disable the firewall or allow all incoming packets that come from LSF hosts.

The bpeek command also uses random ports for communication. Either disable the

firewall or allow all incoming packets that come from LSF hosts.

A simple firewall rule example

This example is based on iptables in Linux and enables network communication for the

bjobs command. The “OUTPUT” chain controls the outgoing packets and the “INPUT”

chain controls the incoming packets that are locally delivered.

1. At a minimum, you should open all ports in the “OUTPUT” chain first.

2. For the “INPUT” chain, open specific ports on non-master hosts listed in Table 3,

and open specific ports on master hosts listed in Table 4.

When you finish configuring the firewall rules in the “INPUT” chain that are described

in Table 3 and Table 4, use “iptables –L INPUT” to check whether you have set the rules

successfully.

Table 3. Ports to open on non-master hosts

Condition Port type Port Configurat

ion file

Protocol

If you configure

LSB_QUERY_PORT

source LSB_QUERY_PORT lsf.conf TCP

If you do not

configure

LSB_QUERY_PORT

source LSB_MBD_PORT lsf.conf TCP

N/A source LSF_LIM_PORT lsf.conf TCP

N/A destination LSF_LIM_PORT lsf.conf TCP

N/A destination LSF_LIM_PORT lsf.conf UDP



Table 4. Ports to open on master hosts

Condition Port type Port Configurat

ion file

Protocol

If you configure

LSB_QUERY_PORT

destination LSB_QUERY_PORT lsf.conf TCP

If you do not

configure

LSB_QUERY_PORT

destination LSB_MBD_PORT lsf.conf TCP

N/A source LSF_LIM_PORT lsf.conf TCP

N/A destination LSF_LIM_PORT lsf.conf TCP

N/A source LSF_LIM_PORT lsf.conf UDP


In a Platform MultiCluster environment

If you only use the firewall to block packets between different clusters, you need to open

only the two ports in Table 5 on the master host. If you also use the firewall between

hosts in the same cluster, follow the “In a single-cluster environment” steps above.

Table 5. Ports between clusters


LSF_LIM_PORT lsf.conf TCP+UDP

LSB_MBD_PORT lsf.conf TCP

Setting up LSF firewall rules in Windows This section describes two ways of setting firewall rules in Windows:

By opening ports

A more strict binary-based method using the built-in Windows firewall

The binary-based firewall method is safer, because it assures that ports can only be used

by specific LSF binaries. The LSF 9.1.3 Windows installer automatically sets binary-based

firewall rules for LSF.


Opening ports in LSF for Windows These rules are similar to setting up LSF firewall rules in Linux. Do not block outgoing

packets from all LSF binaries and allow some specific binaries to accept incoming

packets. The ports that need to be opened are listed in Table 6.

Table 6. LSF ports

Binary name Port Configuration file Protocol lim LSF_LIM_PORT lsf.conf TCP+UDP

mbatchd LSB_MBD_PORT lsf.conf TCP res LSF_RES_PORT lsf.conf TCP

sbatchd LSB_SBD_PORT lsf.conf TCP

If you set LSF_ENABLE_EGO=Y in lsf.conf to enable EGO, open the EGO-related

ports listed in Table 7.

Table 7. EGO-related ports

Binary name Port Configuration file Protocol vemkd EGO_KD_PORT ego.conf TCP pem EGO_PEM_PORT ego.conf TCP

egosc ESC_PORT egosc_conf.xml TCP wsgserver WSG_PORT wsg.conf TCP

named 53 (default port of named) named.conf TCP+UDP

If you do not set the LSF_NIOS_PORT_RANGE environment variable, allow bsub and

lsrun to use any port to get incoming packets.

If you do set LSF_NIOS_PORT_RANGE, allow bsub and lsrun to use the specified

range of ports with the TCP protocol.

The bpeek and pam commands should be allowed to use any port to get incoming

packets. And if you use bpeek command, res also should be allowed to use any port to

get incoming packets.

Set up LSF firewall rules by using Windows firewall with

advanced security The following example applies to the Windows firewall on Windows server 2008 and

later, which allows you to block both inbound and outbound network traffic. You must

configure the following properties for each Windows firewall rule for LSF:

1. The firewall rule name.

2. A full path of the specific program.

3. Profile properties:

Set properties to all active profiles. Windows advanced firewall has three types of

profiles: domain, private, and public. Setting the profile properties to all active

profiles enables LSF to work no matter which profile the firewall currently uses.


4. Network direction properties.

Specify whether the rule is an inbound rule or an outbound rule.

Inbound rules filter traffic passing from the network to the local computer based

on the filtering conditions specified in the rule. If an LSF binary needs to get

incoming information from the network, it needs an inbound firewall rule.

Outbound rules filter traffic passing from the local computer to the network based

on the filtering conditions specified in the rule. An outbound firewall rule is

needed when an LSF program wants to send requests out to the network.

5. Protocol properties and port number properties.

Only the inbound rules for some LSF daemon binaries need these two properties.

Example Windows firewall configuration

An example configuration is shown in Table 8. The example is based on the following

assumptions:

1. This example uses LSF ports defined in lsf.conf (LSF_LIM_PORT, LSF_RES_PORT,

LSB_MBD_PORT and LSB_SBD_PORT). The example assumes the following default

port numbers:

LSF_LIM_PORT=6879

LSB_MBD_PORT=6881

LSB_MBD_PORT=6882 You cannot limit the port number of RES because commands like bpeek use random

ports to exchange information with RES.

2. The example also uses EGO-related ports defined in ego.conf (EGO_KD_PORT and

EGO_PEM_PORT), in egosc_conf.xml (ESC_PORT), and in wsg.conf

(WSG_PORT). The example assumes the following default port numbers:

EGO_KD_PORT=6777

EGO_PEM_PORT=6778

ESC_PORT=6779

WSG_PORT=9090

3. The example assumes that the DNS server named binary uses the default port number

53.

4. Only lim and the named binary need two kinds of inbound firewall rules and the

only difference is the protocol property.

5. If you set LSF_NIOS_PORT_RANGE for NIOS communication, you must set up a

new firewall rule for NIOS whose protocol property is TCP and open the port range

at the firewall accordingly. By default, there is no limit for the port property of the

firewall rule for NIOS and NIOS-related binaries.


Table Key:

Any: the binary can receive network packets with matching any value in

protocol field and port field.

Y: You must set up a new inbound or outbound firewall rule.

N: You do not need to set up a new inbound or outbound firewall rule.

Table 8. Example of LSF firewall rules based on Windows firewall with advanced

security

Binary name Inbound

firewall rule

Outbound

firewall rule

Protocol and port

number for inbound

rules egosc Y Y TCP: 6779 esub.password N Y N/A lim Y Y TCP: 6879

UDP:6879 mbatchd Y Y TCP: 6881 named Y Y TCP: 53

UDP:53 nios Y Y Any pem Y Y TCP: 6778 res Y Y Any sbatchd Y Y TCP: 6882 TSJobHelper Y Y Any vemkd Y Y TCP: 6777 wsgserver Y Y TCP: 9090 bacct N Y N/A badmin N Y N/A bapp N Y N/A bbot N Y N/A bchkpnt N Y N/A bclusters N Y N/A bconf N Y N/A bgadd N Y N/A bgbroker N Y N/A bgdel N Y N/A bgmod N Y N/A bgpinfo N Y N/A bhist N Y N/A bhosts N Y N/A bhpart N Y N/A bjdepinfo N Y N/A bjgroup N Y N/A bjobs N Y N/A bkill N Y N/A blaunch N Y N/A blimits N Y N/A


Binary name Inbound

firewall rule

Outbound

firewall rule

Protocol and port

number for inbound

rules bmg N Y N/A bmgroup N Y N/A bmig N Y N/A bmod N Y N/A bparams N Y N/A bpeek N Y Any bpost N Y N/A bqueues N Y N/A bread N Y N/A brequeue N Y N/A bresize N Y N/A bresources N Y N/A brestart N Y N/A bresume N Y N/A brsvadd N Y N/A brsvdel N Y N/A brsvmod N Y N/A brsvs N Y N/A brun N Y N/A bsla N Y N/A bslots N Y N/A bstatus N Y N/A bstop N Y N/A bsub N Y Any bswitch N Y N/A btop N Y N/A bugroup N Y N/A busers N Y N/A egosh N Y N/A lsadmin N Y N/A lsclusters N Y N/A lseligible N Y N/A lsgrun N Y N/A lshosts N Y N/A lsid N Y N/A lsinfo N Y N/A lsload N Y N/A lsloadadj N Y N/A lslogin N Y N/A lsmail N Y N/A lsmon N Y N/A lspasswd N Y N/A lsplace N Y N/A


Binary name Inbound

firewall rule

Outbound

firewall rule

Protocol and port

number for inbound

rules lsrcp N Y N/A lsrtasks N Y N/A lsrun N Y Any lssmtp N Y N/A pam Y Y Any ppmsetvar N Y N/A TaskStarter N Y N/A tspeek N Y N/A tssub N Y N/A


Best practices

This document has described best practices for the following:

Provides guidelines for setting up LSF firewall rules in Linux.

Provides guidelines for setting up LSF firewall rules in Windows.


Conclusion This document provides guidelines for setting up LSF firewall rules in both Linux and

Windows and provides an integrated example of setting up LSF firewall rules based on

Windows firewall with advanced security.


Further reading Administering Platform LSF Version 9 Release 1.3:

o PDF format: http://publibfp.dhe.ibm.com/epubs/pdf/c2753023.pdf

o IBM Knowledge Center:

http://www.ibm.com/support/knowledgecenter/SSETD4_9.1.3/lsf_kc_ad

min.dita

Contributors

Dianpeng Liang

LSF Architect

Xun Pan

LSF Developer

Xiaofeng Ye

LSF Developer

http://publibfp.dhe.ibm.com/epubs/pdf/c2753023.pdf

http://www.ibm.com/support/knowledgecenter/

http://www.ibm.com/support/knowledgecenter/SSETD4_9.1.3/lsf_kc_admin.dita

http://www.ibm.com/support/knowledgecenter/SSETD4_9.1.3/lsf_kc_admin.dita


Notices This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other

countries. Consult your local IBM representative for information on the products and services

currently available in your area. Any reference to an IBM product, program, or service is not

intended to state or imply that only that IBM product, program, or service may be used. Any

functionally equivalent product, program, or service that does not infringe any IBM

intellectual property right may be used instead. However, it is the user's responsibility to

evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in

this document. The furnishing of this document does not grant you any license to these

patents. You can send license inquiries, in writing, to:

IBM Director of Licensing

IBM Corporation

North Castle Drive

Armonk, NY 10504-1785

U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where

such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES

CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER

EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-

INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do

not allow disclaimer of express or implied warranties in certain transactions, therefore, this

statement may not apply to you.

Without limiting the above disclaimers, IBM provides no representations or warranties

regarding the accuracy, reliability or serviceability of any information or recommendations

provided in this publication, or with respect to any results that may be obtained by the use of

the information or observance of any recommendations provided herein. The information

contained in this document has not been submitted to any formal IBM test and is distributed

AS IS. The use of this information or the implementation of any recommendations or

techniques herein is a customer responsibility and depends on the customer’s ability to

evaluate and integrate them into the customer’s operational environment. While each item

may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee

that the same or similar results will be obtained elsewhere. Anyone attempting to adapt

these techniques to their own environment does so at their own risk.

This document and the information contained herein may be used solely in connection with

the IBM products discussed in this document.

This information could include technical inaccuracies or typographical errors. Changes are

periodically made to the information herein; these changes will be incorporated in new

editions of the publication. IBM may make improvements and/or changes in the product(s)

and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only

and do not in any manner serve as an endorsement of those websites. The materials at those

websites are not part of the materials for this IBM product and use of those websites is at your

own risk.

IBM may use or distribute any of the information you supply in any way it believes

appropriate without incurring any obligation to you.

Any performance data contained herein was determined in a controlled environment.

Therefore, the results obtained in other operating environments may vary significantly. Some

measurements may have been made on development-level systems and there is no

guarantee that these measurements will be the same on generally available systems.

Furthermore, some measurements may have been estimated through extrapolation. Actual

results may vary. Users of this document should verify the applicable data for their specific

environment.


Information concerning non-IBM products was obtained from the suppliers of those products,

their published announcements or other publicly available sources. IBM has not tested those

products and cannot confirm the accuracy of performance, compatibility or any other

claims related to non-IBM products. Questions on the capabilities of non-IBM products should

be addressed to the suppliers of those products.

All statements regarding IBM's future direction or intent are subject to change or withdrawal

without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily business operations. To

illustrate them as completely as possible, the examples include the names of individuals,

companies, brands, and products. All of these names are fictitious and any similarity to the

names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE: © Copyright IBM Corporation 2014. All Rights Reserved.

This information contains sample application programs in source language, which illustrate

programming techniques on various operating platforms. You may copy, modify, and

distribute these sample programs in any form without payment to IBM, for the purposes of

developing, using, marketing or distributing application programs conforming to the

application programming interface for the operating platform for which the sample

programs are written. These examples have not been thoroughly tested under all conditions.

IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these

programs.

Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International

Business Machines Corporation in the United States, other countries, or both. If these and

other IBM trademarked terms are marked on their first occurrence in this information with a

trademark symbol (® or ™), these symbols indicate U.S. registered or common law

trademarks owned by IBM at the time this information was published. Such trademarks may

also be registered or common law trademarks in other countries. A current list of IBM

trademarks is available on the Web at “Copyright and trademark information” at

www.ibm.com/legal/copytrade.shtml

Windows is a trademark of Microsoft Corporation in the United States, other countries, or

both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

Contacting IBM To provide feedback about this paper, write to [email protected].

To contact IBM in your country or region, check the IBM Directory of Worldwide

Contacts at http://www.ibm.com/planetwide

To learn more about IBM Information Management products, go to

http://www.ibm.com/software/data/

mailto:[email protected]?subject=paper's%20Setting%20up%20firewall%20rules%20for%20IBM%20Platform%20LSF

http://www.ibm.com/planetwide

http://www.ibm.com/software/data/

Documents

Best practices - IBM€¦ · Setting up firewall rules for IBM Platform LSF Page 6 of 17 If you set LSF_ENABLE_EGO=Y in lsf.conf to enable Platform EGO, you must open the following