Skip to content

Gen weight refactor manual merge #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kdlong wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Gen weight refactor manual merge #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
cd2a110
First implementation of WeightGroupInfo/product classes
kdlong Jan 15, 2022
222d8c3
Working on parsing integration with scale/pdf weights
kdlong Aug 15, 2019
4cb142f
New WeightGroups, improved parsing with helper class
kdlong Jan 15, 2022
91a7901
updates to allow saving weight sums from all weight categories
sroychow Aug 21, 2020
8f101c5
Add more error handling
Oct 14, 2020
09380dd
adding altset index table
sroychow Nov 3, 2020
14b1034
Use cms::Exception, configure debugging, code formatting
kdlong Jan 16, 2022
4fc34fd
attempt to delete extra kets after the last </weightgroup>
SanghyunKo Nov 20, 2020
10b0115
Step forward to newer cmssw version
kdlong Jan 16, 2022
28d0302
Allow gen products to run at GEN step or Nano
kdlong Jan 16, 2022
85a4fe7
Support ignoregroups in nano, fixes for unassociated weights
kdlong Jan 3, 2022
0d58e0f
Convert OwnVector to unique_ptr
kdlong Jan 3, 2022
eacce78
Simplify nano producer and weight parsing
kdlong Jan 16, 2022
a29eea3
Make producers edm::Global
kdlong Jan 16, 2022
359187b
Code format, don't fail for missing LHEEventProduct
kdlong Jan 16, 2022
d3d2af5
Remove redundant files (renamed or dropped)
kdlong Jan 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Configuration/StandardSequences/python/Generator_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
#
from PhysicsTools.HepMCCandAlgos.genParticles_cfi import *
from GeneratorInterface.Core.generatorSmeared_cfi import *
from GeneratorInterface.Core.genWeights_cfi import genWeights
from GeneratorInterface.Core.lheWeights_cfi import lheWeights
from RecoJets.Configuration.RecoGenJets_cff import *
from RecoMET.Configuration.RecoGenMET_cff import *
from RecoJets.Configuration.GenJetParticles_cff import *
Expand Down Expand Up @@ -53,10 +55,11 @@
VertexSmearing = cms.Sequence(cms.SequencePlaceholder("VtxSmeared"))
GenSmeared = cms.Sequence(generatorSmeared)
GeneInfo = cms.Sequence(GeneInfoTask)
genWeightsSeq = cms.Sequence(genWeights*lheWeights)
genJetMET = cms.Sequence(genJetMETTask)

from SimPPS.Configuration.GenPPS_cff import *
pgen = cms.Sequence(cms.SequencePlaceholder("randomEngineStateProducer")+VertexSmearing+GenSmeared+GeneInfo+genJetMET, PPSTransportTask)
pgen = cms.Sequence(cms.SequencePlaceholder("randomEngineStateProducer")+VertexSmearing+GenSmeared+GeneInfo+genWeightsSeq+genJetMET, PPSTransportTask)

# sequence for bare generator result only, without vertex smearing and analysis objects added

Expand Down
4 changes: 4 additions & 0 deletions DataFormats/NanoAOD/src/classes_def.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<lcgdict>
<class name="nanoaod::FlatTable::Column" ClassVersion="3">
<version ClassVersion="3" checksum="3066258528"/>
Expand All @@ -10,6 +11,9 @@
<class name="nanoaod::FlatTable::RowView" transient="true" />
<class name="edm::Wrapper<nanoaod::FlatTable>" />

<class name="std::vector<nanoaod::FlatTable>" />
<class name="edm::Wrapper<std::vector<nanoaod::FlatTable>>" />

<class name="nanoaod::MergeableCounterTable::FloatColumn" ClassVersion="3">
<version ClassVersion="3" checksum="828208870"/>
</class>
Expand Down
12 changes: 12 additions & 0 deletions GeneratorInterface/Configuration/python/GeneratorInterface_EventContent_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@
'keep GenLumiInfoProduct_generator_*_*',
'keep GenEventInfoProduct_generator_*_*',
'keep edmHepMCProduct_generatorSmeared_*_*',
'keep GenWeightProduct_lheWeights_*_*',
'keep GenWeightProduct_genWeights_*_*',
'keep GenWeightInfoProduct_lheWeights_*_*',
'keep GenWeightInfoProduct_genWeights_*_*',
'keep GenFilterInfo_*_*_*',
'keep *_genParticles_*_*'
)
Expand All @@ -34,6 +38,10 @@
'keep GenLumiInfoProduct_generator_*_*',
'keep GenEventInfoProduct_generator_*_*',
'keep edmHepMCProduct_generatorSmeared_*_*',
'keep GenWeightProduct_lheWeights_*_*',
'keep GenWeightProduct_genWeights_*_*',
'keep GenWeightInfoProduct_lheWeights_*_*',
'keep GenWeightInfoProduct_genWeights_*_*',
'keep GenFilterInfo_*_*_*',
'keep *_genParticles_*_*'
)
Expand All @@ -48,6 +56,10 @@
'keep GenLumiInfoHeader_generator_*_*',
'keep GenLumiInfoProduct_generator_*_*',
'keep GenEventInfoProduct_generator_*_*',
'keep GenWeightProduct_genWeights_*_*',
'keep GenWeightInfoProduct_lheWeights_*_*',
'keep GenWeightInfoProduct_genWeights_*_*',
'keep GenFilterInfo_*_*_*',
'keep GenFilterInfo_*_*_*',
'keep *_genParticles_*_*'
)
Expand Down
1 change: 1 addition & 0 deletions GeneratorInterface/Core/BuildFile.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
<use name="lhapdf"/>
<use name="f77compiler"/>
<use name="root"/>
<use name="tinyxml2"/>
<export>
<lib name="1"/>
</export>
27 changes: 27 additions & 0 deletions GeneratorInterface/Core/interface/GenWeightHelper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#ifndef GeneratorInterface_Core_GenWeightHelper_h
#define GeneratorInterface_Core_GenWeightHelper_h

#include <string>
#include <vector>
#include <map>
#include <regex>
#include <fstream>

#include "SimDataFormats/GeneratorProducts/interface/PdfWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/ScaleWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/PartonShowerWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/GenLumiInfoProduct.h"
#include "GeneratorInterface/Core/interface/WeightHelper.h"

#include <tinyxml2.h>

namespace gen {
class GenWeightHelper : public WeightHelper {
public:
GenWeightHelper();
std::vector<std::unique_ptr<gen::WeightGroupInfo>> parseWeightGroupsFromNames(std::vector<std::string> weightNames,
bool addUnassociatedGroup) const;
};
} // namespace gen

#endif
53 changes: 53 additions & 0 deletions GeneratorInterface/Core/interface/LHEWeightHelper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#ifndef GeneratorInterface_Core_LHEWeightHelper_h
#define GeneratorInterface_Core_LHEWeightHelper_h

#include <string>
#include <vector>
#include <map>
#include <regex>
#include <fstream>

#include "SimDataFormats/GeneratorProducts/interface/UnknownWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/MEParamWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/PdfWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/ScaleWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/LHERunInfoProduct.h"
#include "GeneratorInterface/Core/interface/WeightHelper.h"

#include <tinyxml2.h>

namespace gen {
class LHEWeightHelper : public WeightHelper {
public:
LHEWeightHelper() : WeightHelper(){};

enum class ErrorType { Empty, SwapHeader, HTMLStyle, NoWeightGroup, TrailingStr, Unknown, NoError };
const std::unordered_map<ErrorType, std::string> errorTypeAsString_ = {
{ErrorType::Empty, "Empty header"},
{ErrorType::SwapHeader, "Header info out of order"},
{ErrorType::HTMLStyle, "Header is invalid HTML"},
{ErrorType::TrailingStr, "Header has extraneous info"},
{ErrorType::Unknown, "Unregonized error"},
{ErrorType::NoError, "No error here!"}};

std::vector<std::unique_ptr<gen::WeightGroupInfo>> parseWeights(std::vector<std::string> headerLines,
bool addUnassociated) const;
bool isConsistent(const std::string& fullHeader) const;
void swapHeaders(std::vector<std::string>& headerLines) const;
void setFailIfInvalidXML(bool value) { failIfInvalidXML_ = value; }
bool failIfInvalidXML() const { return failIfInvalidXML_; }

private:
std::string weightgroupKet_ = "</weightgroup>";
std::string weightTag_ = "</weight>";
bool failIfInvalidXML_ = false;
std::string parseGroupName(tinyxml2::XMLElement* el) const;
ParsedWeight parseWeight(tinyxml2::XMLElement* inner, std::string groupName, int groupIndex, int& weightIndex) const;
bool validateAndFixHeader(std::vector<std::string>& headerLines, tinyxml2::XMLDocument& xmlDoc) const;
tinyxml2::XMLError tryReplaceHtmlStyle(tinyxml2::XMLDocument& xmlDoc, std::string& fullHeader) const;
tinyxml2::XMLError tryRemoveTrailings(tinyxml2::XMLDocument& xmlDoc, std::string& fullHeader) const;
ErrorType findErrorType(int xmlError, const std::string& headerLines) const;
};
} // namespace gen

#endif
142 changes: 142 additions & 0 deletions GeneratorInterface/Core/interface/WeightHelper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
#ifndef GeneratorInterface_LHEInterface_WeightHelper_h
#define GeneratorInterface_LHEInterface_WeightHelper_h

#include "SimDataFormats/GeneratorProducts/interface/GenWeightProduct.h"
#include "SimDataFormats/GeneratorProducts/interface/GenWeightInfoProduct.h"
#include "SimDataFormats/GeneratorProducts/interface/WeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/PdfWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/WeightsInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/UnknownWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/PdfWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/PartonShowerWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/ScaleWeightGroupInfo.h"
#include "SimDataFormats/GeneratorProducts/interface/MEParamWeightGroupInfo.h"
#include "LHAPDF/LHAPDF.h"
#include <boost/algorithm/string.hpp>
#include <bits/stdc++.h>
#include <fstream>
#include <memory>

namespace gen {
struct ParsedWeight {
std::string id;
int index;
std::string groupname;
std::string content;
std::unordered_map<std::string, std::string> attributes;
int wgtGroup_idx;
};

class WeightHelper {
public:
WeightHelper();

template <typename T>
std::unique_ptr<GenWeightProduct> weightProduct(const GenWeightInfoProduct& weightsInfo,
std::vector<T> weights,
float w0) const;

void setGuessPSWeightIdx(bool guessPSWeightIdx) {
PartonShowerWeightGroupInfo::setGuessPSWeightIdx(guessPSWeightIdx);
}
void addUnassociatedGroup(std::vector<std::unique_ptr<gen::WeightGroupInfo>>& weightGroups) const {
gen::UnknownWeightGroupInfo unassoc("unassociated");
unassoc.setDescription("Weights with missing or invalid header meta data");
weightGroups.push_back(std::make_unique<gen::UnknownWeightGroupInfo>(unassoc));
}
int addWeightToProduct(GenWeightProduct& product, double weight, std::string name, int weightNum, int groupIndex);
void setDebug(bool value) { debug_ = value; }

protected:
bool debug_ = false;
const unsigned int FIRST_PSWEIGHT_ENTRY = 2;
const unsigned int DEFAULT_PSWEIGHT_LENGTH = 46;
std::map<std::string, std::string> currWeightAttributeMap_;
std::map<std::string, std::string> currGroupAttributeMap_;
bool isScaleWeightGroup(const ParsedWeight& weight) const;
bool isMEParamWeightGroup(const ParsedWeight& weight) const;
bool isPdfWeightGroup(const ParsedWeight& weight) const;
bool isPartonShowerWeightGroup(const ParsedWeight& weight) const;
bool isOrphanPdfWeightGroup(ParsedWeight& weight) const;
void updateScaleInfo(gen::ScaleWeightGroupInfo& scaleGroup, const ParsedWeight& weight) const;
void updateMEParamInfo(const ParsedWeight& weight, int index) const;
void updatePdfInfo(gen::PdfWeightGroupInfo& pdfGroup, const ParsedWeight& weight) const;
void updatePartonShowerInfo(gen::PartonShowerWeightGroupInfo& psGroup, const ParsedWeight& weight) const;
void cleanupOrphanCentralWeight(WeightGroupInfoContainer& weightGroups) const;
bool splitPdfWeight(ParsedWeight& weight, WeightGroupInfoContainer& weightGroups) const;

int lhapdfId(const ParsedWeight& weight, gen::PdfWeightGroupInfo& pdfGroup) const;
std::string searchAttributes(const std::string& label, const ParsedWeight& weight) const;
std::string searchAttributesByTag(const std::string& label, const ParsedWeight& weight) const;
std::string searchAttributesByRegex(const std::string& label, const ParsedWeight& weight) const;

// Possible names for the same thing
const std::unordered_map<std::string, std::vector<std::string>> attributeNames_ = {
{"muf", {"muF", "MUF", "muf", "facscfact"}},
{"mur", {"muR", "MUR", "mur", "renscfact"}},
{"pdf", {"PDF", "PDF set", "lhapdf", "pdf", "pdf set", "pdfset"}},
{"dyn", {"DYN_SCALE"}},
{"dyn_name", {"dyn_scale_choice"}},
{"up", {"_up", "Hi"}},
{"down", {"_dn", "Lo"}},
{"me_variation", {"mass", "sthw2", "width"}},
};
void printWeights(const WeightGroupInfoContainer& weightGroups) const;
std::unique_ptr<WeightGroupInfo> buildGroup(ParsedWeight& weight) const;
WeightGroupInfoContainer buildGroups(std::vector<ParsedWeight>& parsedWeights, bool addUnassociatedGroup) const;
std::string searchString(const std::string& label, const std::string& name) const;
};

template <typename T>
std::unique_ptr<GenWeightProduct> WeightHelper::weightProduct(const GenWeightInfoProduct& weightsInfo,
std::vector<T> weights,
float w0) const {
auto weightProduct = std::make_unique<GenWeightProduct>(w0);
weightProduct->setNumWeightSets(weightsInfo.numberOfGroups());
gen::WeightGroupData groupData = {0, nullptr};
int i = 0;
// size=1 happens if there are no PS weights, so the weights vector contains only the central GEN weight.
if (weights.size() > 1) {
// This gets remade every event to avoid having state-dependence in the helper class
// could think about doing caching instead
int unassociatedIdx = weightsInfo.unassociatedIdx();
std::unique_ptr<gen::UnknownWeightGroupInfo> unassociatedGroup;
if (unassociatedIdx != -1)
unassociatedGroup = std::make_unique<gen::UnknownWeightGroupInfo>("unassociated");
for (const auto& weight : weights) {
double wgtval;
std::string wgtid;
if constexpr (std::is_same<T, gen::WeightsInfo>::value) {
wgtid = weight.id;
wgtval = weight.wgt;
} else if (std::is_same<T, double>::value) {
wgtid = std::to_string(i);
wgtval = weight;
}
try {
groupData = weightsInfo.containingWeightGroupInfo(i, groupData.index);
} catch (const cms::Exception& e) {
if (unassociatedIdx == -1)
throw e;
if (debug_) {
std::cout << "WARNING: " << e.what() << std::endl;
}
// Access the unassociated group separately so it can be modified
unassociatedGroup->addContainedId(i, wgtid, wgtid);
groupData = {static_cast<size_t>(unassociatedIdx), unassociatedGroup.get()};
}
int entry = groupData.group->weightVectorEntry(wgtid, i);

// TODO: is this too slow?
if (debug_)
std::cout << "Adding weight num " << i << " EntryNum " << entry << " to group " << groupData.index
<< std::endl;
weightProduct->addWeight(wgtval, groupData.index, entry);
i++;
}
}
return weightProduct;
}
} // namespace gen

#endif
1 change: 1 addition & 0 deletions GeneratorInterface/Core/plugins/BuildFile.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<use name="FWCore/MessageLogger"/>
<use name="FWCore/Utilities"/>
<use name="SimDataFormats/GeneratorProducts"/>
<use name="GeneratorInterface/Core"/>
<use name="FWCore/SharedMemory"/>
<use name="clhep"/>
<library name="GeneratorInterfaceCore_plugins" file="*.cc">
Expand Down
Loading