GOkb style Master list

[iyawnis]

"252 = The ‘master’ list is a package which contains TIPPs for all the TIs known to be supplied by a specific Content Provider - that is, it is a list compiled by combining all the other lists we know of for that provider. Since the same TI will appear in multiple packages for the same Provider, these will need de-duplicating. Earliest start date across all TIPPs used as Start Date on Master package TIPP. Latest end date across all TIPPs used as End Date on Master package TIPP. "

Is this something that will be produced on-demand? E.g A screen where the user will input the provider of the master list, and then an export is created? Or is this an automatic process, that will generate one package for every provider in the system, and keep it updated with a scheduled overnight task?

And to make sure I got the right idea: We create a new 'master' package for each provider. For every TitleInstance associated with an existing TIPP for that provider, we have a new TIPP keeping any associations and properties same, but Package will be the master list.
As you said, we might have the same TI on several TIPPs. What are the criteria for deciding which TIPP details to keep ('these will need de-duplicating')?

[ostephens]

This is something that needs to be created and updated with a scheduled task.

I think that rather than keeping the details from one of the TIPPs it would be preferable to:

• Use the coverage start date/vol/issue from the TIPP with the earliest start date
• Use the coverage end date/vol/issue from the TIPP with the latest end date

However, it is not clear to me what should happen with other properties - e.g. coverage notes. We may want to look at how GOKb handles this as the brief was to do this in a similar way (but there was some question of whether GOKb could end up with multiple TIPPs for the same TI, and was made clear that this wasn't something that KB+ wanted).

The GOKb code can be tracked back from https://github.com/k-int/gokb-phase1/blob/dev/server/gokb/grails-app/jobs/org/gokb/MasterListUpdateJob.groovy. @sosguthorpe wrote this I think so may be able to advise on any logic used in terms of selecting TIPP properties for the Master list

[sosguthorpe]

From memory, I think we go through each package associated with a
particular Org and copy each TIPP but change the package to the "Master
package" for the current Org. So all TIPP properties would be duplicated
except the package.

On 02/08/15 23:38, Owen Stephens wrote:

This is something that needs to be created and updated with a
scheduled task.

I think that rather than keeping the details from one of the TIPPs it
would be preferable to:

• Use the coverage start date/vol/issue from the TIPP with the
  earliest start date
• Use the coverage end date/vol/issue from the TIPP with the latest
  end date

However, it is not clear to me what should happen with other
properties - e.g. coverage notes. We may want to look at how GOKb
handles this as the brief was to do this in a similar way (but there
was some question of whether GOKb could end up with multiple TIPPs for
the same TI, and was made clear that this wasn't something that KB+
wanted).

The GOKb code can be tracked back from
https://github.com/k-int/gokb-phase1/blob/dev/server/gokb/grails-app/jobs/org/gokb/MasterListUpdateJob.groovy.
@sosguthorpe https://github.com/sosguthorpe wrote this I think so
may be able to advise on any logic used in terms of selecting TIPP
properties for the Master list

—
Reply to this email directly or view it on GitHub
#993 (comment).

[ostephens]

Thanks @sosguthorpe. Does that mean in GOKb the Master package has multiple TIPPs for the same TI? That is the thing that the KB+ data managers have said they don't want

[sosguthorpe]

Had to consult the code because I couldn't remember :)

When a master TIPP is created it is linked to the TIPP on which it is
based. However, if the current TIPP has no master associated with it we
check for an existing one, in the correct master package, that has the
same TI and Platform. If we find one then we update that one and link to
this TIPP too. That means that the last none master TIPP that has the
same TI and Platform will be the one used to set the properties on the
master. Typing this has made me realise that this is actually probably
not what we need now since the extension of the TIPP properties. We
should probably be using earliest start date and latest end dates etc.

In short there is one TIPP per TI on the master and that Master TIPP
could be linked to multiple regular TIPPs. The property settings
probably need to be revisited now that we have extended what we know
about the TIPPs.

On 03/08/15 10:16, Owen Stephens wrote:

Thanks @sosguthorpe https://github.com/sosguthorpe. Does that mean
in GOKb the Master package has multiple TIPPs for the same TI? That is
the thing that the KB+ data managers have said they /don't/ want

—
Reply to this email directly or view it on GitHub
#993 (comment).

[ostephens]

Thanks @sosguthorpe

[iyawnis]

Looking at the code, I think there are no duplicated tipps, line 257 in
PackageService. It looks to me that all of the logic can be transferred
over.

On 3 August 2015 at 10:16, Owen Stephens notifications@github.com wrote:

Thanks @sosguthorpe https://github.com/sosguthorpe. Does that mean in
GOKb the Master package has multiple TIPPs for the same TI? That is the
thing that the KB+ data managers have said they don't want

—
Reply to this email directly or view it on GitHub
#993 (comment).

[ostephens]

@latusaki based on @sosguthorpe's last comment may need to enhance the logic slightly?

[iyawnis]

Yeah, change the behaviour on what gets copied over to the master tipp.

[ostephens]

In terms of other fields:

tipp_access_start
tipp_access_end

should follow same logic as tipp_start_date/tipp_end_date (i.e. take the earliest/latest - and null on the tipp_access_end/tipp_end_date always taken as the latest date)

coverage_depth should always be fulltext.

The fields I'm not sure what the rules should be are:

coverage_note
tipp_embargo
tipp_delayedoa
tipp_hybridoa
tipp_payment

I think it was this issue that led us to have links between the master tipp and other tipps in GOKb - so we didn't need to merge this stuff together

[sosguthorpe]

My personal opinion would be to favour the less restrictive values (if
that way of thinking can be applied to the property). So hybridoa a
value of "Yes" would carry the most weight.

On 03/08/15 11:22, Owen Stephens wrote:

In terms of other fields:

tipp_access_start
tipp_access_end

should follow same logic as tipp_start_date/tipp_end_date (i.e. take
the earliest/latest - and null on the tipp_access_end/tipp_end_date
always taken as the latest date)

The fields I'm not sure what the rules should be are:

coverage_note
tipp_embargo
tipp_delayedoa
tipp_hybridoa
tipp_payment

I think it was this issue that led us to have links between the master
tipp and other tipps in GOKb - so we didn't need to merge this stuff
together

—
Reply to this email directly or view it on GitHub
#993 (comment).

[iyawnis]

Does this mean for a master package, all tipps will have the same properties? Same start/end dates access start /end etc? Are there any properties that we need to keep from the original TIPP when creating the master TIPP other than the title instance?