Conversation
|
Hey @dylanw-oss 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
| // | ||
| // val processedData = featurizedModel.transform(convertedLabelDataset).select(colstoSelect.map(col): _*) | ||
|
|
||
| val processedData = featurizedModel.transform(convertedLabelDataset).select(getFeaturesCol, getLabelCol) |
There was a problem hiding this comment.
This line of the code is to unblock the cache and fit afterwards.
There is a performance issue here, it will cache the whole dataset, but we only need the "features" and "label" column.
There was a problem hiding this comment.
But the change will make another problem, if user set WeightCol, it will throw exception that weightCol is not available.
I tried to add it above (the commented code) but it's not working,
if (isDefined(weightCol) || !$(weightCol).isEmpty)
this will throw exception, java.util.NoSuchElementException: Failed to find a default value for weightCol
| println(s"$this - [trainInternal] residualsDF1 columns ${residualsDF1.columns.mkString(",")}, size ${residualsDF1.count()}") | ||
| println(s"$this - [trainInternal] residualsDF2 columns ${residualsDF2.columns.mkString(",")}, size ${residualsDF2.count()}") | ||
|
|
||
| val coefficients = Array(residualsDF1, residualsDF2).map(regressor.fit).map(_.coefficients(0)) |
There was a problem hiding this comment.
Based on the logs, DML notebook run is being blocked here.
Any idea?
3 participants
Related Issues/PRs
#xxx
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.yarn run startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.