Optimizer Service (OPT)
Overview
The Optimizer Service provides a way for java programs to access the same software engine used in the Northfield Optimizer Desktop software. It is designed to provide an object model that follows the same tree structure as that used by the desktop software. This enables analysts and programmers to share a common model for defining and implementing analytics projects. Technically, the OptimizerService provides a java API on top of a system that performs the required mappings and transfer to and from the underlying C++ engine via the java native interface (JNI). The C++ engine is contained in a system library that is included in the CCS_BASE area.
OPT Operation
Scenario Development
The Northfield Optimizer is designed as a multi purpose tool that supports user scenarios starting with simple risk analyses, portfolio optimizations based on user provided parameters, through to complex multiple optimizations that can estimate parameters. As a multi purpose tool with complex settings that does not constrain the ability of the user to explore their data, there is a high risk of user error in the configuration of inputs and interpretation of outputs. In order to minimize and manage errors, we recommend your user interface:
- Implements clearly defined scenarios that are prototyped in the desktop tool.
- Ensures datasets are present, complete and correct for the User scenario
- Ensures input values and parameters are within the ranges you have prototyped
- Only presents the User with the specific join/run function required for your scenario
- Produces output reports and tables that are relevant to the scenario.
It is beyond the scope of this document to describe the operations of the OptimizerService in detail, and the best way to determine the correct settings for an optimizer project is to create a desktop project and then map the fields you wish to set by referencing the GUI form names and mapping them to the same objects in the API Javadoc. Note that the Java API does not provide access to any of the file based methods of the underlying C++ API.
The Northfield support team is available to help you develop and test scenario prototypes, and the sample programs demonstrate how to organise and start an optimizer project.
Error Handling
Clients need to be informed as to what occurred inside an optimization process. The OptimizerService provides access to both optimizer messages and error flags with typed error codes.
Error Messages
The optimizer messages are the text messages that appear in the window at the bottom of the Windows GUI application. Java programs can access these messages using the following methods:
// After a successful project run:
String[] messages = project.reports().messages();
// After an OptimizerException is thrown:
String[] messages = exception.messages();
Error Flags
The service layer pattern used by the NISASP defines an abstract programmer model that isolates the client user program from the underlying implementation model. Java programs that interface to libraries via the JNI have a mismatch between the information free integer error reporting in the native library called in the JNI library and the typed exception model used in java APIs. The NISASP implements a hybrid model to resolve the mismatch that occurs between the OptimizerService that calls the C++ optimizer library via the JNI. The hybrid model implementation maps the integer constants defined in the C++ error structures to integer values mapped to Java enumerated types. This mapping is performed in the NisoptEngine plugin that plugs the C++ library into the OptimizerService.
The flow of errors codes is:
- The C++ library returns one or more flags to the JNI program defined in C structs.
- The JNI program converts C struct values to integer values mapped in the NisoptEngine.Error table.
- The JNI program returns the integer values to the OptimizerService which attaches the codes to the OptimizerException returned to the User Program via the method: int[] OptimizerException.flags(); The exception is marked by the JNI library using an Exception type defined by the OptimizerException.Type.
- The User Program queries the NisoptEngine.Error table for the enumerated type that corresponds to the flag values.
This process is shown in the following diagram.
The following sample shows how to map exception flags to NisoptEngine codes.
try {
...
OptimizerService opt = ccs.get(OptimizerService.class);
Project project = opt.create();
opt.run(project, ProjectTask.RUN);
} catch (OptimizerException ex) {
System.out.println("Service Exception:");
Type type = ex.type();
System.out.println(" " + type.name() + ":" + ex.getMessage());
System.out.println("Engine Flags:");
for (int flag : ex.flags()) {
NisoptEngine.Error err = NisoptEngine.Error.get(flag);
System.out.println(" " + flag + ":" + err.name());
}
System.out.println("Engine Messages:");
Tools.printLog(ex.messages(), " ");
} ...
The output of this program after a failed join is:
=====================================
Optimizer Error Tests
=====================================
Service Exception:
|DATA_ERROR:join failed
Engine Flags:
617:ERR_JOIN_MODEL
Engine Messages:
Not enough data for JOIN
Join failure
Error Flag Types
The OptimizerException.Flag is an error category flag that can be accessed from:
OptimizerException.Flag OptimizerException.flag();
The type flags correspond to the JNI process of:
- copying data over (|DATA_ERROR),
- validation before processing (SETTINGS_ERROR),
- errors that occur during processing (PROCESS_ERROR), and
- an internal failure (ENGINE_ERROR).
| Exception Type | Code | Text |
|---|---|---|
| USAGE_ERROR | 100 | API usage exception |
| DATA_ERROR | 200 | |
| SETTINGS_ERROR | 300 | settings/configuration error |
| PROCESS_ERROR | 400 | process error. Eg. unable find solution |
| ENGINE_ERROR | 500 | internal error |
Error Flag Values
The NisoptEngine.Error table is a lookup table that can be used to find an enumerated type that corresponds to an integer flag value returned from:
int[] OptimizerException.flags();
For the NisoptEngine, more than one flag of PROCESS_ERROR type can be returned, for all other types only one flag is expected to be returned.
The enumerated type name is derived from the C++ struct name used in the JNI library.
| Exception Type | Enumerated Type Name | Code | Text |
|---|---|---|---|
| DATA_ERROR | ERR_JOIN_ZEROPRICEFOUND | 601 | |
| DATA_ERROR | ERR_JOIN_ZEROPORTVALUE | 602 | |
| DATA_ERROR | ERR_JOIN_EMPTYBENCHMARK | 603 | |
| DATA_ERROR | ERR_JOIN_COMPASS | 604 | |
| DATA_ERROR | ERR_JOIN_PORTFILE | 605 | |
| DATA_ERROR | ERR_JOIN_BUYFILE | 606 | |
| DATA_ERROR | ERR_JOIN_BENCHFILE | 607 | |
| DATA_ERROR | ERR_JOIN_DBFILE | 608 | |
| DATA_ERROR | ERR_JOIN_ALPHAFILE | 609 | |
| DATA_ERROR | ERR_JOIN_MINFILE | 610 | |
| DATA_ERROR | ERR_JOIN_MAXFILE | 611 | |
| DATA_ERROR | ERR_JOIN_MINTRADESIZEFILE | 612 | |
| DATA_ERROR | ERR_JOIN_ROUNDBASEFILE | 613 | |
| DATA_ERROR | ERR_JOIN_TRANSBUY | 614 | |
| DATA_ERROR | ERR_JOIN_TRANSSELL | 615 | |
| DATA_ERROR | ERR_JOIN_PRICEFILE | 616 | |
| DATA_ERROR | ERR_JOIN_MODEL | 617 | |
| DATA_ERROR | ERR_JOIN_CORR | 618 | |
| DATA_ERROR | ERR_JOIN_QPEN | 619 | |
| DATA_ERROR | ERR_JOIN_MODELB | 620 | |
| DATA_ERROR | ERR_JOIN_CORRB | 621 | |
| DATA_ERROR | ERR_JOIN_IND | 622 | |
| DATA_ERROR | ERR_JOIN_SECT | 623 | |
| DATA_ERROR | ERR_JOIN_MTBL | 624 | |
| DATA_ERROR | ERR_JOIN_COMPASSETS | 625 | |
| DATA_ERROR | ERR_JOIN_ACC | 626 | |
| DATA_ERROR | ERR_JOIN_XML_SOURCE | 627 | |
| DATA_ERROR | ERR_JOIN_INDMAP | 628 | |
| DATA_ERROR | ERR_JOIN_NLTC_CORR | 629 | |
| DATA_ERROR | ERR_JOIN_REFPORT | 630 | |
| DATA_ERROR | ERR_JOIN_NLTC_SYS | 631 | |
| DATA_ERROR | ERR_JOIN_RESIDUAL | 632 | |
| DATA_ERROR | ERR_JOIN_THRESHOLD | 633 | |
| DATA_ERROR | ERR_JOIN_UCITS | 634 | |
| SETTINGS_ERROR | ERR_JOIN | 635 | error during JOIN |
| SETTINGS_ERROR | ERR_RUN | 636 | error during RUN |
| SETTINGS_ERROR | ERR_RAPZERO | 637 | RAP is 0 |
| SETTINGS_ERROR | ERR_NEGATIVEVAL | 638 | negative variance in the factor table |
| SETTINGS_ERROR | ERR_WRONGMINMAX | 639 | wrong min max constraints |
| SETTINGS_ERROR | MSG_MAXITERREACHED | 640 | maximum number of iterations reached |
| SETTINGS_ERROR | MSG_MAXTURNOVERREACHED | 641 | maximum turnover value is reached |
| SETTINGS_ERROR | MSG_MAXTURNOVERREACHED1 | 642 | maximum turnover value is reached |
| SETTINGS_ERROR | MSG_CANTFINDABYBESTSWAP | 643 | cannot find any best stock to buy or sell |
| SETTINGS_ERROR | MSG_MAXPRESREACHED | 644 | maximum precision is reached |
| SETTINGS_ERROR | MSG_THRESHOLDERROR | 645 | cannot find any swap to solve pairing problem |
| SETTINGS_ERROR | MSG_MAXGAPGAINREACHED | 646 | maximum capital gain value is reached |
| PROCESS_ERROR | OPT_VIOLATIONS_SECURITY | 647 | violation of min/max constraints for stock |
| PROCESS_ERROR | OPT_VIOLATIONS_GROUP | 648 | industry constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_SECTOR | 649 | sector constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_FACTOR | 650 | factor constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_THRESHOLD | 651 | threshold constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_MINTRADESIZE | 652 | minimum trade size constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_MAXASSETS | 653 | max assets constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_LONGSHORT | 654 | long/short constraints are violated |
| PROCESS_ERROR | OPT_VIOLATIONS_MAXTRACKINGERROR | 655 | maximum tracking error constraints are violated |
| ENGINE_ERROR | ERR_WRONGINPUTSTR | 656 | |
| ENGINE_ERROR | ERR_COMPLETECALC | 657 | |
| ENGINE_ERROR | ERR_MEMALLOC | 658 | |
| ENGINE_ERROR | ERR_TWOSPACE1 | 659 | |
| ENGINE_ERROR | ERR_TWOSPACE2 | 660 | |
| ENGINE_ERROR | ERR_TWOSPACE3 | 661 | |
| ENGINE_ERROR | ERR_TWOSPACE4 | 662 | |
| ENGINE_ERROR | ERR_UNDEFSTEPCODE | 663 | |
| ENGINE_ERROR | MSG_WRONGVARVALUE | 664 | |
| ENGINE_ERROR | ERR_TWOSPACE1_5 | 665 | |
| ENGINE_ERROR | MSG_CANCELEDBYUSER | 666 | |
| ENGINE_ERROR | MSG_INITPROBLEMFAULT | 667 | |
| ENGINE_ERROR | ERR_USERBEFOREOPT | 668 | |
| ENGINE_ERROR | ERR_STOPEED_BYCALLBACK_AFTER_EACH_ITER | 669 | |
| ENGINE_ERROR | ERR_UNDEFINED | 670 |
OPT Configuration
All configuration is normally managed by ConnectionService mode defaults.
Notes
- Percentage Values: Asset Weights and percentages are the same as the desktop and should be set as values between 0 - 100 where 100 = 100%. File Operations
- All file and GUI based operations are disabled in the OptimizerService. This includes the file methods in the C++ API. The reason for this is that the OptimizerService component is designed to be clustered and file based operations bind an instance to a specific machine configuration.