For one of my experiments I needed to develop a PGX client in Java, connection to a PGX server and performing a bunch of actions.
I remember that in the Tutorials page of PGX there was a full tutorial on how to build a PGX client application with Maven, which could make my life easier.
What is the benefit of Maven here? All the JARs required by the PGX client itself. For example if you download the PGX 3.2.0 Java Client ZIP the lib folder has 31 JAR files: some are Oracle own PGX specific libraries, others are public available packages.
It’s absolutely fine to manage all that by hand, to add all these JARs in the buildpath and manage things like that. Maven provides an even easier solution: a single dependency in your project and everything else is taken care automatically. No need to care or look which version of which file is used.
No more Oracle Maven for PGX
Sadly for me the tutorial didn’t work! After some debugging and making sure there wasn’t an authentication issue with the Oracle Maven repo, I posted on the ODC forum seeking for feedbacks (the thread is here: https://community.oracle.com/thread/4205246).
Six minutes later I got the answer …
Good catch: we are not publishing new versions of the client to that maven repo any more due to licensing reasons, we should have removed that page from the documentation from version 3.2.0.
We will remove it.
Well …
Not really helping me, I will need to switch back and manage all the dependencies by hand? Having to always extract all the files in the right place and make sure the buildpath is set correctly?
Sorry but although not being a Maven expert I believe there must be another way to do it!
Adding files to a local Maven repository
Over the 31 JARs included in the downloaded PGX 3.2.0 Java Client ZIP file, 5 are custom Oracle PGX packages, 26 are public available packages.
There are definitely a bunch of these which aren’t required for the client itself, but I’m not going to go there for now.
Google helped a lot in telling me how to import a JAR file into the local Maven repository and how to define the various dependencies.
Here are the steps to get Maven managing the PGX 3.2.0 Client packages for you automatically:
Import the 5 Oracle PGX JARs
First step is to add 4 of the 5 files without any special dependency, just load the JARs into the local Maven repository.
From the folder containing the JARs of the PGX client run these commands:
|
1 2 3 4 5 6 7 |
mvn install:install-file -Dfile=pgx-api-3.2.0.jar -DgroupId=oracle.pgx -DartifactId=pgx-api -Dversion=3.2.0 -Dpackaging=jar mvn install:install-file -Dfile=pgx-common-3.2.0.jar -DgroupId=oracle.pgx -DartifactId=pgx-common -Dversion=3.2.0 -Dpackaging=jar mvn install:install-file -Dfile=pgx-shell-3.2.0.jar -DgroupId=oracle.pgx -DartifactId=pgx-shell -Dversion=3.2.0 -Dpackaging=jar mvn install:install-file -Dfile=graph-query-ir-2019-01-04_4.jar -DgroupId=oracle.pgx -DartifactId=graph-query-ir -Dversion=2019-01-04_4 -Dpackaging=jar |
Names used for groupID, artifactId and version follow the tutorial which wasn’t working and the Maven standards.
Define dependencies in a POM file
Once these files are there the important step is to import the last one, the pgx-client-3.2.0.jar, and inform Maven of all the dependencies it has: the 4 JARs just imported above and the 26 coming from public online repositories (which Maven will download automatically when required).
All these things are defined into a POM file, in this case I named it pgx-client-3.2.0.pom as it’s the POM file (Project Object Model, an XML file being the fundamental unit of work in Maven) of that package.
To make it easier for you, here is the content of the file. Listing all the JARs included in the ZIP you download from the Oracle website.
Create a file named pgx-client-3.2.0.pom next to the pgx-client-3.2.0.jar.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 |
<?xml version="1.0" encoding="UTF-8"?> <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <modelVersion>4.0.0</modelVersion> <groupId>oracle.pgx</groupId> <artifactId>pgx-client</artifactId> <version>3.2.0</version> <description>POM was created by hand</description> <url>https://www.oracle.com/technetwork/oracle-labs/parallel-graph-analytix/downloads/index.html</url> <dependencies> <!-- ### Oracle PGX manually added to Maven repo files ### --> <dependency> <!-- pgx-api-3.2.0.jar --> <groupId>oracle.pgx</groupId> <artifactId>pgx-api</artifactId> <version>3.2.0</version> </dependency> <dependency> <!-- pgx-common-3.2.0.jar --> <groupId>oracle.pgx</groupId> <artifactId>pgx-common</artifactId> <version>3.2.0</version> </dependency> <dependency> <!-- pgx-shell-3.2.0.jar --> <groupId>oracle.pgx</groupId> <artifactId>pgx-shell</artifactId> <version>3.2.0</version> </dependency> <dependency> <!-- graph-query-ir-2019-01-04_4.jar --> <groupId>oracle.pgx</groupId> <artifactId>graph-query-ir</artifactId> <version>2019-01-04_4</version> </dependency> <!-- ### Public available JARs in Maven ### --> <dependency> <groupId>aopalliance</groupId> <artifactId>aopalliance</artifactId> <version>1.0</version> </dependency> <dependency> <groupId>net.sourceforge.argparse4j</groupId> <artifactId>argparse4j</artifactId> <version>0.8.1</version> </dependency> <dependency> <groupId>commons-codec</groupId> <artifactId>commons-codec</artifactId> <version>1.11</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.5</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.8.1</version> </dependency> <dependency> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> <version>1.2</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-vfs2</artifactId> <version>2.2</version> </dependency> <dependency> <groupId>it.unimi.dsi</groupId> <artifactId>fastutil</artifactId> <version>8.1.1</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>fluent-hc</artifactId> <version>4.5.6</version> </dependency> <dependency> <groupId>org.codehaus.groovy</groupId> <artifactId>groovy-all</artifactId> <version>2.4.15</version> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>26.0-jre</version> </dependency> <dependency> <groupId>com.google.inject</groupId> <artifactId>guice</artifactId> <version>4.2.2</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.6</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpcore</artifactId> <version>4.4.10</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-annotations</artifactId> <version>2.9.7</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-core</artifactId> <version>2.9.7</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.9.8</version> </dependency> <dependency> <groupId>org.fusesource.jansi</groupId> <artifactId>jansi</artifactId> <version>1.17.1</version> </dependency> <dependency> <groupId>javax.inject</groupId> <artifactId>javax.inject</artifactId> <version>1</version> </dependency> <dependency> <groupId>jline</groupId> <artifactId>jline</artifactId> <version>2.14.6</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.11.0</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.11.0</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-jcl</artifactId> <version>2.11.0</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-slf4j-impl</artifactId> <version>2.11.0</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.25</version> </dependency> <dependency> <groupId>io.swagger</groupId> <artifactId>swagger-annotations</artifactId> <version>1.5.18</version> </dependency> </dependencies> </project> |
Once the POM file is created, in the same folder as the pgx-client-3.2.0.jar file, the last step is to import this missing JAR with the POM file into the local Maven repository:
|
1 |
mvn install:install-file -Dfile=pgx-client-3.2.0.jar -DpomFile=pgx-client-3.2.0.pom |
No need to care about dependencies anymore…
I can now add a PGX 3.2.0 client into any Maven project simply by adding a dependency on :
groupId=oracle.pgx
artifactId=pgx-client
version=3.2.0
Every other JAR will be automatically provided by Maven either by downloading them online or loading them from the local repository (which also contains the 5 PGX JARs not existing in any online repository).
No need to say it but …
Remember you can’t publish these 5 JARs, which aren’t available, in a public Maven repository online. There are licenses not allowing you to make that code public.
From the comments…
Davide Basilio Bartolini posted a great script in the comments. By mixing bash and python it looks into the JARs contained in the PGX Java Client ZIP and check if they comes with a POM file. If they do, the script extract the groupId, artifactId and version directly from there. This allows you to easily write your own pgx-client-3.2.0.pom file telling Maven all the dependencies (and searching online for those not found).
No need to say it works for any version if you update the value of PGX_VERSION in it.
As the formatting in comments is very limited and the script is 100% helpful I post it back here in full text (tried to attach a .sh file, WordPress got scared it could be harmful and kindly denied …).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#!/bin/bash PGX_VERSION=3.2.0 unzip pgx-${PGX_VERSION}-java-client.zip for lib in pgx-${PGX_VERSION}/lib/*.jar; do pom_file=$(unzip -Z1 $lib | grep 'pom.xml$') unzip -p $lib $pom_file |\ sed 's/<project xmlns=.*>/<project>/'|\ python3 <(cat <<EOS import sys import xml.etree.ElementTree as ET try: tree = ET.parse(sys.stdin) project = tree.getroot() groupId = project.find('groupId').text artifactId = project.find('artifactId').text version = project.find('version').text dep = ''' <dependency> <groupId>{}</groupId> <artifactId>{}</artifactId> <version>{}</version> </dependency> '''.format(groupId, artifactId, version) print(dep) except: sys.stderr.write('Warning: Cannot find pom info for {}\n'.format(sys.argv[1])) EOS ) $lib done |
Very nice post, thanks Gianni!
I put together a small bash + python script that can help to generate the dependencies for the pom.xml:
#!/bin/bashPGX_VERSION=3.2.0
unzip pgx-${PGX_VERSION}-java-client.zip
for lib in pgx-${PGX_VERSION}/lib/*.jar; do
pom_file=$(unzip -Z1 $lib | grep 'pom.xml$')
unzip -p $lib $pom_file |\
sed 's///'|\
python3 <(cat <<EOS
import sys
import xml.etree.ElementTree as ET
try:
tree = ET.parse(sys.stdin)
project = tree.getroot()
groupId = project.find('groupId').text
artifactId = project.find('artifactId').text
version = project.find('version').text
dep = '''
{}
{}
{}
'''.format(groupId, artifactId, version)
print(dep)
except:
sys.stderr.write('Warning: Cannot find pom info for {}\n'.format(sys.argv[1]))
EOS
) $lib
done
It doesn’t work for all the jars, unfortunately, because some of them don’t include a pom.xml themselves, but I hope it can help.
And maybe it can be extended to handle the missing jars as well, by looking elsewhere in the metadata.
If only I had that yesterday … would have saved me 26 mvnrepository.com searches 😀
Do you maybe happen to have the real list of JARs required for the Java PGX client only? (so without the shell, groovy etc.)
Off the top of my head, I think that the jars only required by the shell would be:
* pgx-shell-3.2.0.jar
* groovy-all-2.4.15.jar
* jline-2.14.6.jar
I would have to double check, but I think all other jars are needed for the java client.
The script is posted at the end of the post above comments to give it the formatting it deserves (and because WordPress dislike .sh files as attachments)