Array filter fails if zero results pass the predicate

[davesheldon]

Describe the bug
I want to assert that a particular jsonpath evaluates to an empty array. This instead returns jsonpath.ErrorMemberNotExist.

To Reproduce

package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$[?(@.id == 2)].name`, `[ { "id": 1, "name": "One" } ]`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

This returns: jsonpath.ErrorMemberNotExist, member did not exist (path=[?(@.id == 2)])

Expected behavior
I expected it to return an empty array/slice (e.g. any[]{}).

[davesheldon]

For now, I'm working around this by using an aggregate function and making 2 queries:

query 1: $[*].length()
query 2: $[?(@.id != 2)].length()

If they're equal then it isn't in the array, but there are still edge cases where it can fail (e.g. if my assertion is wrong, or the array is empty to begin with).

[AsaiYusuke]

Dave,

Thank you for your feedback. I have looked into the issue you reported and have observed similar behavior where jsonpath.ErrorMemberNotExist is returned in the following section of the test code:

$[?(@.a==5)]

Furthermore, there are tests in the json-path-comparison repository, which my library is based on, that execute similar queries. My library follows the behavior of Not found consensus.

• Original: Filter expression with equals on array without match
• Results from running the tests in my repository: Filter expression with equals on array without match

As an another workaround, you may consider checking the type of the returned value and overwrite output. Here's an example in Go:

    output, err := jsonpath.Retrieve(jsonPath, src)
    if err != nil {
        switch err.(type) {
        case jsonpath.ErrorMemberNotExist:
            output = []interface{}{}
        default:
            fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
            return
        }
    }

[davesheldon]

Thanks for the quick reply. I think my issue with your workaround is that it isn't clear whether MemberNotExist describes a missing property or just an empty array. My implementation works such that I'm passing in a jsonpath expression from a file, so I can't tell whether I should be passing along an error for a missing member or returning an empty array.

If there were a dedicated NotFound type for array filter functions I could definitely key off of that, though. Worst case I can use your workaround and make my users figure out on their own why they got [] as a result in those other cases.

[AsaiYusuke]

If there are additional issues regarding this matter, it's important to note that the JSONPath presented from you raises another concern.
If Retrieve were able to handle empty results in the middle of JSONPath in filter, attempting to check .name on an empty result would trigger a different error.

	jsonPath, srcJSON := `$.name`, `[]`

	jsonpath.ErrorTypeUnmatched, type unmatched (expected=object, found=[]interface {}, path=.name)

Therefore, in the sample code, not only should cases where the filter doesn't match be treated as [], but also any reference to name on [] should be considered as [].

[davesheldon]

Yes, I'm aware of that. I wrote the example as a toy demonstration because I didn't want to clutter it up with the aggregate function I'm actually trying to use there. The actual expression I'm hoping to use is more like:

$[?(@.id == 2)].someAggregate()

[AsaiYusuke]

In this library, the Function syntax doesn't anticipate empty arrays and is designed to operate on elements. I'm aware that this might not be the most user-friendly approach.

Here's an example, albeit not a perfect one. When I tested it with Python's built-in functions, I encountered a similar element-centric implementation.

#!python

a = [1]
print(max(a))
a = []
print(max(a))

1
Traceback (most recent call last):
  File "/python-test.py", line 6, in <module>
    print(max(a))
ValueError: max() arg is an empty sequence

[davesheldon]

I think that’s reasonable enough. My main issue at this point is that the error MemberNotExist isn’t specific enough to really determine what the issue was (invalid property or empty array).

[davesheldon]

But for now I’ve taken your suggestion and am just returning [] for both kinds of errors. If you want to change this then great, otherwise I’ll work around it :) thanks so much for the dialogue!

[AsaiYusuke]

Thank you for sharing the situation. I appreciate your patience, and while I've been pondering for a few days, I haven't come up with a clean solution to handle empty arrays while maximizing compatibility. If I ever come across a better approach, I'll definitely give it a shot.