IG S3 API

Problem Statement:

File-based processing has been one of the common integration patterns in every implementation of Guidewire for InsuranceSuite Applications. By introducing the “Integration Framework,” clients are more hesitant to use the OOTB Inbound/outbound framework, striving to externalize the transformation login into the IG Apps. IGS3API provides a generic solution that IS, vendors or clients' internal applications use to manipulate files. They can upload, read, and delete a file and merge files in a specific prefix into a single batch file. In addition, IS Admin pages get enhanced to support the development team with their investigations or validations of the files stored in the S3 bucket prefixes. They allow them to view the file's metadata and content or list the files. (for this enhancement, we can leverage the Admin UI pages for the Inbound/Outbound framework )

Persona / Use Case(s):

Development and QA teams will benefit from this as a standard component. The enhanced UI supporting the S3 bucket will benefit developers, QAs, leads, and admin users. Admin UIs constantly improve troubleshooting and investigation when something goes wrong.

Solution and Business Value:

Here is the list of endpoints published by IGS3API: - Upload file: a header determines the behavior to override or error out if the file already exists - Read file: a header determines the behavior to error out or return empty if the file doesn't exist - Delete file: a header determines the behavior to error out or ignore if the file doesn't exist - Merge files (resembles the outbound file framework) : - Combine the contents of all the files located in a prefix into one file - Moves the successfully processed files to the processed prefix based on the request - Moves the faulty records to the failed prefix based on the request - Creating a history JSON file capturing the merge details such as: - Name of the merged files - Process Time - Total number of records failed or successfully processed - List of the access keys for the failed files - Merging files processes all the files located in a prefix, providing pagination considering the performance and guarantee delivery (FYI, there is a limitation of 1000 records for aws also; gws3 doesn't support all component options like max-keys) -The request for this provides enough directions on where to read the files, where to move the files after processing, where to store the history records, etc. - Transfer file to SFTP server - Retrieve the file from the SFTP server. - Expand file: - Reads a file and splits it into records - A history record gets stored in the specified location, showing the progress/result - The original file gets moved to the archived prefix - Expand the file and process (resembling Inbound File Framework) - Reads a file and splits it into records - A history record gets stored in the specified location, showing the progress/result - Redirects each file's content to the specified endpoint for processing - A history record gets stored in the specified location, showing the progress of the processing of the records - Failed records moved to a specified prefix (failed prefix) - Processes records moved to a specified prefix (processed prefix) - The original file gets moved to the archived prefix

File-based integration is a typical pattern used by almost all IS implementations. A standard approach addressing common aspects of the programming/development, from error handling, logging, and profiling to performance and guarantee delivery, would save us time during development and is one step towards the ready-to-market goal. This IG app is for IS applications and clients integrating with IG Apps via the S3 bucket. IGS3API encapsulates all the operations we do on the files in the S3 bucket. Having a robust component in IG that can provide the same functionality as the inbound/outbound framework aligns with the Integration Framework approach.

Additional Information:

I can share the code for most of the operations if required.