You can generate Certificate in java dynamically, by using a pair or keys. (Public Key, Private Keys). Get These keys as BigInteger format and checking the following code to generate certificate.
RSAPrivateKeySpec serPrivateSpec = new RSAPrivateKeySpec(new BigInteger(val of pub key), new BigInteger(val of pri key));
fact = KeyFactory.getInstance("RSA");
PrivateKey serverPrivateKey = fact.generatePrivate(serPrivateSpec);
RSAPublicKeySpec serPublicSpec = new RSAPublicKeySpec(new BigInteger(agentCL.getSerPubMod()), new BigInteger(agentCL.getSerPubExp()));
PublicKey serverPublicKey = fact.generatePublic(serPublicSpec);
keyStore = KeyStore.getInstance(IMXAgentCL.STORE_TYPE);
keyStore.load(null, SOMEPWD.toCharArray());
Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
X509Certificate[] serverChain = new X509Certificate[1];
X509V3CertificateGenerator serverCertGen = new X509V3CertificateGenerator();
X500Principal serverSubjectName = new X500Principal("CN=OrganizationName");
serverCertGen.setSerialNumber(new BigInteger("123456789"));
// X509Certificate caCert=null;
serverCertGen.setIssuerDN(somename);
serverCertGen.setNotBefore(new Date());
serverCertGen.setNotAfter(new Date());
serverCertGen.setSubjectDN(somename);
serverCertGen.setPublicKey(serverPublicKey);
serverCertGen.setSignatureAlgorithm("MD5WithRSA");
// certGen.addExtension(X509Extensions.AuthorityKeyIdentifier, false,new
// AuthorityKeyIdentifierStructure(caCert));
serverCertGen.addExtension(X509Extensions.SubjectKeyIdentifier, false,
new SubjectKeyIdentifierStructure(serverPublicKey));
serverChain[0] = serverCertGen.generateX509Certificate(serverPrivateKey, "BC"); // note: private key of CA
keyStore.setEntry("xyz",
new KeyStore.PrivateKeyEntry(serverPrivateKey, serverChain),
new KeyStore.PasswordProtection("".toCharArray()));
Hope this will help you.