Nuclear Gradients by maxscheurer · Pull Request #124 · adc-connect/adcc

maxscheurer · 2021-01-13T16:10:56Z

Implemented together with @Drrehn.

Features

ToDo

lgtm-com · 2021-01-13T16:37:13Z

This pull request introduces 1 alert when merging 6b92a09 into 6e7ac34 - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-01-13T17:39:00Z

This pull request introduces 1 alert when merging 7136d03 into 6e7ac34 - view on LGTM.com

new alerts:

1 for Unused local variable

adcc/AmplitudeVector.py

mfherbst · 2021-01-14T10:44:33Z

Awesome 👍

lgtm-com · 2021-01-16T12:13:38Z

This pull request introduces 1 alert when merging 844cc1c into 6e7ac34 - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-01-16T12:46:05Z

This pull request introduces 1 alert when merging 10980bf into 6e7ac34 - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-01-17T10:07:39Z

This pull request introduces 1 alert when merging 3f82618 into 6e7ac34 - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-01-19T10:48:03Z

This pull request introduces 1 alert when merging b92ee1b into 041cdcb - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-02-21T09:37:28Z

This pull request introduces 1 alert when merging 1a3c802 into 041cdcb - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-06-03T13:18:16Z

This pull request introduces 1 alert when merging 54a9181 into e4ca2ed - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2021-12-14T15:21:33Z

This pull request introduces 4 alerts when merging 92e7535 into e4ca2ed - view on LGTM.com

new alerts:

3 for Unused import
1 for Unused local variable

lgtm-com · 2021-12-14T18:44:27Z

This pull request introduces 4 alerts when merging df675fa into e4ca2ed - view on LGTM.com

new alerts:

3 for Unused import
1 for Unused local variable

mfherbst · 2021-12-15T08:46:26Z

Of course tests need to work and all, but I wonder if a good first aim is to just get the things already implemented merged into master and release a new version?

maxscheurer · 2021-12-15T11:06:37Z

The tests are horrible to implement... ensuring highly converged eigenvectors is difficult 😞 but in principle all the implemented gradient methods are correct 👍🏻

iubr · 2021-12-15T12:07:24Z

Not sure if this is a good suggestion, but maybe we could run a couple of small molecules and save the results in a text file for pytest to compare agains (this is more or less how it's done in veloxchem). I can try out the cvs-adc gradients and find an optimal/good enough setup (I guess the question is of comparing analytical vs. numerical gradients)

maxscheurer · 2021-12-15T12:11:11Z

The generation of finite-difference reference data works fine already, for both generic and CVS-ADC 😊👍🏻

mfherbst · 2021-12-15T12:54:10Z

The tests are horrible to implement... ensuring highly converged eigenvectors is difficult

Can't you lower to tolerance or introduce perturbations in the molecule / orbitals to ensure they are sufficiently distinct (to avoid rotations in invariant subspaces etc.)

maxscheurer · 2021-12-15T12:57:08Z

You mean the convergence tolerance/atol for asserts?

You mean geometry perturbations to break symmetries? Yes, I can try that for sure 👍🏻

mfherbst · 2021-12-15T12:58:30Z

You mean the convergence tolerance/atol for asserts?

Yes. It's not amazing, but it will at least catch forgotten factors of two or switched signs.

maxscheurer · 2021-12-17T17:03:38Z

@iubr, the CVS-ADC2-x don't seem to be correct right now, at least I get an error for H2O of ~1e-5 (compared to 5-point finite differences). Maybe I made some mistake while cleaning up the code... Which accuracy could you obtain in your tests? I'll try to run your original branch again at some point to get to the bottom of this.

iubr · 2021-12-18T13:26:47Z

The ~1e-5 is similar to what I get with my code (in comparison to the 2-point finite difference numerical gradient). I checked it today with a 5-point finite difference and it is a little smaller, but still same order (1e-5). So if there is a bug, it would be in my module as well.
I had discussed this difference with Patrick and we had concluded that it could be due to the tolerance/convergence threshold for the adc step (I couldn't go better than 1e-6 because of the requirement for the HF reference). Of course, this might not be the right explanation for the difference. It's, of course, possible that there is still an undiscovered bug or mistake somewhere in the code/equations. I'll make another check next week. Let's see.

maxscheurer · 2021-12-18T14:37:41Z

I had discussed this difference with Patrick and we had concluded that it could be due to the tolerance/convergence threshold for the adc step

I converge the SCF to 1e-12, and the ADC procedure is converged to 1e-10 in the energy. Given the fact that all other gradient methods so far can achieve at least 1e-7 accuracy, I'm convinced that one can achieve the same for CVS-ADC2-x . I'll try to add canonical ADC2-x gradients now and see how accurate one can get.

iubr · 2021-12-19T00:44:32Z

Thanks a lot!

iubr · 2022-02-13T23:41:44Z

Thanks! If you'd like an overview of the density matrices, they are from https://doi.org/10.1063/5.0058221, Appendix C, Table II, 3rd column (ADC matrix terms).

iubr · 2022-02-21T07:04:08Z

Hi Max,

I just found the bug! It was a sign in one of the terms for the right-hand-side of the CV block, lambda multipliers. See here.
I tried to merge the latest version of adcc/gradients branch into my cvs_gradients branch, but for some reason I cannot get it running on my computer, so I fixed the bug using the old version of the code. If you could make the fix at your end and test cvs-adc2x again, it would be super great!! Thanks a lot!! I'll make a few more tests on my end just to make sure there's nothing else.

/Iulia

maxscheurer · 2022-02-21T08:04:57Z

That's great news! I will try it out later. 👍🏻😊

iubr · 2022-02-21T11:27:30Z

Thank you!! Let me know how it goes :D
(I still haven't figured out the puzzle of the factor 2 in the VVVV block between my module and adcc, but I'm planning to look into it again tomorrow)

maxscheurer · 2022-02-21T14:28:08Z

I found the reason for the weird factor 2 behavior in this
block of code. The blocks oooo and vvvv were added again in the case of CVS, such that they existed twice in the TwoParticleDensityMatrix object. 😁

I found some other inconsistencies in the formation of g2a_hf and g2a_oresp, with these fixes altogether, everything works!!! 💪

mfherbst · 2022-02-21T15:36:53Z

Cool! Great work.

iubr · 2022-02-21T23:12:32Z

Wonderful ^-^!! Thanks a lot @maxscheurer!!

mfherbst

Just a few comments ... obviously I have not at all checked the equations.

mfherbst · 2022-03-02T17:48:23Z

adcc/test_workflow.py

+        # with pytest.raises(InputError):
+        #     obtain_guesses_by_inspection(matrix1, n_guesses=40, kind="any")


Could change this to a test for the warning.

mfherbst · 2022-03-02T17:49:52Z

adcc/testdata/dump_fdiff_gradient.py

+molecules = [
+    Molecule("h2o", 0, 1),
+]


Suggested change

molecules = [

Molecule("h2o", 0, 1),

]

molecules = [Molecule("h2o")]

adcc/testdata/dump_fdiff_gradient.py

mfherbst · 2022-03-02T17:57:46Z

adcc/backends/psi4.py

+        g2_ao_1, g2_ao_2: relaxed two-particle density matrices
+        """
+        natoms = self.mol.natom()
+        Gradient = {}


Why capitalised?

Shamelessly copied from some psi4numpy tutorial, will change this 😄

mfherbst · 2022-03-02T17:58:41Z

adcc/backends/psi4.py

+        # Build Total OEI Gradient
+        Gradient["OEI"] = Gradient["T"] + Gradient["V"] + Gradient["S"]
+
+        # Build TE contributions


TEI contributions?

mfherbst · 2022-03-04T16:33:03Z

adcc/gradients/__init__.py

+@dataclass(frozen=True)
+class GradientComponents:
+    natoms: int
+    nuc: np.ndarray


Nuc is a bit short, maybe add a comment here to explain

Good idea, I think it would be good to make all attribute names a bit more verbose.

mfherbst · 2022-03-04T16:33:21Z

adcc/gradients/__init__.py

+    nuc: np.ndarray
+    overlap: np.ndarray
+    hcore: np.ndarray
+    two_electron: np.ndarray


comment: Two electron integrals?

adcc/gradients/__init__.py

mfherbst · 2022-03-04T16:37:15Z

adcc/gradients/__init__.py

+        return self.excitation_or_mp.reference_state
+
+    @property
+    def _energy(self):


Why the underscore? If someone uses it it's sort of their fault, no? If you add a warning I think that's fine.

Good point. Will rename it to correlation_energy or so to make it more clear what exactly is returned.

mfherbst · 2022-03-04T16:46:15Z

adcc/gradients/orbital_response.py

+    lam = conjugate_gradient(A, rhs=rhs, x0=x0, Pinv=Pinv,
+                             explicit_symmetrisation=None,
+                             callback=default_print)


Which tolerance is used and why? Should for sure be related to the convergence used for the ADC excitation vectors ... or maybe aim for relative residual decrease?

I think it should rather be related to the SCF convergence criterion, because we compute the response of the HF ground state (kind of 😄). In addition, MP states don't have an additional convergence criterion, so I will make this depend on the SCF conv tol. 👍

fix dot product issue for bare tensors add working gradient implementation add gradient provider to pyscf flake fix excitation view tests

lgtm and coveralls (hopefully) more fdiff fix

hijacked AmplitudeVector for CVS-ADC orbital response. CVS-ADC0 working! cvs-adc1 working

added cvs-adc2 2PDMs,started orbital response (need to check if the RHS is correct) cvs-adc2 gradient working. cvs-adc2x working

… vs. cvs-adc rhs. fixed 1/sqrt(2) additional comments, RHS switched to using symmetrise and antisymmetrise to construct density matrices

mfherbst reviewed Jan 14, 2021

View reviewed changes

adcc/AmplitudeVector.py Outdated Show resolved Hide resolved

maxscheurer force-pushed the gradients branch from 3f82618 to b92ee1b Compare January 19, 2021 10:23

maxscheurer force-pushed the gradients branch 2 times, most recently from 1717c34 to 54a9181 Compare June 3, 2021 12:54

maxscheurer force-pushed the gradients branch 2 times, most recently from 6e67a9d to 365cd5a Compare June 3, 2021 13:57

maxscheurer force-pushed the gradients branch from f3cd5f2 to c16ff2b Compare December 14, 2021 21:32

maxscheurer force-pushed the gradients branch from c16ff2b to d52ac16 Compare December 15, 2021 14:21

maxscheurer force-pushed the gradients branch 2 times, most recently from 06ee1b2 to 51115ed Compare February 21, 2022 15:10

mfherbst reviewed Mar 4, 2022

View reviewed changes

maxscheurer force-pushed the gradients branch from 300f1ba to ef50af9 Compare May 6, 2023 08:04

maxscheurer force-pushed the gradients branch from ef50af9 to beb082d Compare August 25, 2024 12:40

maxscheurer and others added 16 commits October 23, 2024 22:22

getting started on gradients with psi4

db8001b

fix dot product issue for bare tensors add working gradient implementation add gradient provider to pyscf flake fix excitation view tests

add basis gradient tests

5e2e7b3

lgtm and coveralls (hopefully) more fdiff fix

started cvs-adc0 gradient

a72f71d

hijacked AmplitudeVector for CVS-ADC orbital response. CVS-ADC0 working! cvs-adc1 working

added the cvs-adc2 1PDMs

7e5c14d

added cvs-adc2 2PDMs,started orbital response (need to check if the RHS is correct) cvs-adc2 gradient working. cvs-adc2x working

cleaned up code; fixed a few TODOs; remaining: 1/sqrt2, antisymm, adc…

87599e7

… vs. cvs-adc rhs. fixed 1/sqrt(2) additional comments, RHS switched to using symmetrise and antisymmetrise to construct density matrices

tweak test infrastructure

678a6d4

try to finally make damn tests work :(

8fb606c

flake 8 happy?

999ef3f

all molecules hate me, just h2o for now

8fccd86

refactor gradient results, add test for correlation energy

1ff8973

add adc2x gradients, fix cvs-adc2x energy stuff

6ba9bc5

adc3 amplitude response

854d459

correct densities

299ee5b

adc3 gradients working

f718601

warn instead of fail

60444a7

check for warning

65bfcc3

maxscheurer force-pushed the gradients branch from c1c0db5 to 65bfcc3 Compare October 23, 2024 20:23

merge master

164b41e

		# with pytest.raises(InputError):
		# obtain_guesses_by_inspection(matrix1, n_guesses=40, kind="any")

Comments

Conversation

maxscheurer commented Jan 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features

ToDo

Uh oh!

lgtm-com bot commented Jan 13, 2021

Uh oh!

lgtm-com bot commented Jan 13, 2021

Uh oh!

Uh oh!

mfherbst commented Jan 14, 2021

Uh oh!

lgtm-com bot commented Jan 16, 2021

Uh oh!

lgtm-com bot commented Jan 16, 2021

Uh oh!

lgtm-com bot commented Jan 17, 2021

Uh oh!

lgtm-com bot commented Jan 19, 2021

Uh oh!

lgtm-com bot commented Feb 21, 2021

Uh oh!

lgtm-com bot commented Jun 3, 2021

Uh oh!

lgtm-com bot commented Dec 14, 2021

Uh oh!

lgtm-com bot commented Dec 14, 2021

Uh oh!

mfherbst commented Dec 15, 2021

Uh oh!

maxscheurer commented Dec 15, 2021

Uh oh!

iubr commented Dec 15, 2021

Uh oh!

maxscheurer commented Dec 15, 2021

Uh oh!

mfherbst commented Dec 15, 2021

Uh oh!

maxscheurer commented Dec 15, 2021

Uh oh!

mfherbst commented Dec 15, 2021

Uh oh!

maxscheurer commented Dec 17, 2021

Uh oh!

iubr commented Dec 18, 2021

Uh oh!

maxscheurer commented Dec 18, 2021

Uh oh!

iubr commented Dec 19, 2021

Uh oh!

iubr commented Feb 13, 2022

Uh oh!

iubr commented Feb 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maxscheurer commented Feb 21, 2022

Uh oh!

iubr commented Feb 21, 2022

Uh oh!

maxscheurer commented Feb 21, 2022

Uh oh!

mfherbst commented Feb 21, 2022

Uh oh!

iubr commented Feb 21, 2022

Uh oh!

mfherbst left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

maxscheurer commented Jan 13, 2021 •

edited

Loading

iubr commented Feb 21, 2022 •

edited

Loading