Logging vs. Exceptions vs. Result Objects

[dbwiddis]

Early in OSHI's development, the code was littered with UnsupportedOperationExceptions. This seems an appropriate response when we're asking for data that simply doesn't exist (e.g., load average on Windows). But it requires the user to explicitly handle these exceptions.

I moved away from exceptions into log messages, which allowed a more finely grained control of what was a normal problem vs. failure (warnings vs. errors etc.) and just returned sensible defaults (negative values, or 0, or empty strings, or empty collections). This introduces the opposite problem, of not allowing the user to handle exception types.

OSHI needs a standardized method of handling these types of situations:

• Data is undefined on a platform (Windows load avg.)
• Data is not easily obtained on a platform (Windows open files)
• Data requires elevated permissions and/or software installs that the user doesn't have (lots of Linux stuff)
• Data should normally return but happened to fail in this case (Sensor readings) but the user can try again
• Data wasn't updated because you asked for it too recently (tick counts < 1 second apart).

We can use Optional results in key places, or encapsulate the result in our own OshiResult class that includes:

• Result object
• Result type (similar to how VARIANT is used in JNA)
• Result timestamp
• Result error value/enum/etc.

[dblock]

I like this problem. Curious why you didn't specialize exceptions @dbwiddis?

It feels to me that if I am, for example, in need of elevated permissions to get X, I should get an AccessDenied exception. That said, X may be part of a parent object Y, and I should only get the exception when I do Y.X, and not when I get Y.

[cilki]

The concept of API extensions for different platforms like @YoshiEnVerde suggested solves the UnsupportedOperationException problem because it would be impossible to call a method that is unsupported in the first place. For example you would receive a DiskWindows object that can't do things that a DiskLinux can do.

When there's a permissions error or missing software, then the OshiResult can contain the error or a relevant exception can be thrown. I'm not sure which I think is better yet.

[dbwiddis]

Curious why you didn't specialize exceptions @dbwiddis?

As you may recall when you recruited me, I'm an amateur/hobbyist programmer without formal training and with a limited skill set. I didn't even know I could specialize exceptions when I got started. :)

This project has been a great learning experience for me and now that I know better.

For example you would receive a DiskWindows object that can't do things that a DiskLinux can do.

But that removes the platform-independent API that is the core of the project. I want my CentralProcessor interface to have a getLoadAverage() method without knowing or caring which operating system is returning the result. And thus, CentralProcessorWindows must implement this method and do something with it, either returning a default value or empty or null array or throwing an exception, or returning an Optional with no value present.

[cilki]

But that removes the platform-independent API

Both CpuWindows and CpuLinux would extend Cpu which contains features defined on all platforms. If the user knows what OS they are on and wants platform-specific features, they can ask for a CpuLinux. Otherwise they will get a Cpu by default which is platform-independent. (Under the hood, the Cpu instance will really be a CpuLinux)

Since Windows is missing load average, it unfortunately cannot be included in the base type Cpu without reintroducing the problem we are trying to solve. This could be mitigated by creating groups of similar platforms like CpuUnix. That way the user at most needs to know they are on some kind of *nix in order to ask for the load average.

[cilki]

This is basically how it would look. If we strictly follow the "Cpu contains only attributes defined on all platforms" principle, then there are no special cases.

[YoshiEnVerde]

With one of the objectives for this version being decoupling the fetching from the API, we need to think of both parts with different design eyes.

With OSHI trying to both have a small footprint and being performant (which, most of the time, are mutually exclusive things), we can use this to tweak details for the best.

For example: Using immutable objects is great for concurrency and security, but a nightmare on the memory resources, even with caching mitigating things a bit.

The opposite is also true: A very common solution for Java systems that need to keep resource consumption to a minimum is to offload the costs from the memory to the CPU. This is done by saving all the data in the cheapest representation possible, then casting/parsing the data every single time a read/write op is requested.
For example, when we fetch some info from a Linux command (e.g.: dmidecode), instead of asking for a specific field, we can ask for every single field that request can give us in one go, then convert the whole string result into a byte[]. Whenever a request is made for one of the values returned by that call, we parse the contents of the byte[] accordingly.

This second approach is of note for all our info sources that involve parsing OS command results. Specially so, since we could also keep a cache in the next layer for the parsed values themselves, and the data updates will be manually triggered.

[dblock]

+1 for using basic inheritance (@cilki's example with CPU) to avoid having fields that just don't exist on a specific OS

[dbwiddis]

I'm not so fond of using inheritance to solve this problem, since Java doesn't support multiple inheritance and it's a more finely grained issue than OS, and exceptions are common.

Windows doesn't have load average, so in the example above, we create a *nix parent. But macOS doesn't have Disk Queue Length (so do we create a parent of all OS's except Mac for the Disks?)

Mac doesn't have IOWait and IRQ information, and Windows doesn't have Nice and IOWait, so should we have different array sizes here? (or is returning "0" okay as we currently do?) Some stuff is just insanely slow on Windows (counting file handles) so we omit it, while letting the user know it's just "moderately" slow on *nix systems.

Some features are version-dependent (particularly newer vs. older Windows versions or 32- vs. 64-bit) or language dependent, or dependent (especially on *nix) on whether the user has root priviliges (e.g., dmidecode) or has installed a software package (e.g., lshal).

There's stuff that we'd like to put in (Windows Services) that has more information (running/stopped) than the equivalent in other OS's (which are just a list of text strings and don't correlate to process IDs).

Then there's the Windows Sensor stuff, that pretty much relies on OpenHardwareMonitor running in the background, which can in theory be turned on and off while OSHI is running. If it's on, we can return values for some sensors, but what do we return if we only have temperature but not fan speed? If it's off (or OHM doesn't detect fans) do we re-generate a new object with different inheritance that doesn't have a getFanSpeeds() method? When do we switch back? Do we insist on "no third party" programs and just tell users we don't do Windows sensors? How about information we can get from *nix systems if X is running, but can't if it's not?

I could go on, but the point is, we can't solve all of these problems with inheritance and if we do OS-specific code we violate DRY a lot. We have to have a way have a common command across most OS's that we can have a standardized "not on this OS" response.

[dbwiddis]

The gist of my above comment is that I'd prefer:

• The existing common interface, and individual OS classes. If a single OS doesn't have something common to all the other OS's (e.g., Disk queue length or load average) throw an UnsupportedOperationException (or similar).
  • I'm leaning in OSHI 4 to a serializable POJO "Data Object" as a top-level inheritance for the class variables we plan to return, from which platform specific classes inherit (sometimes with an additional abstract class layer). As well as a class-independent "Util" class that does calculations (e.g., for the CentralProcessor the data object would maintain current ticks+timestamp and previous ticks+timestamp, while the util class would have the getCpuUsageBetweenTicks() method.
• Individual functions/methods on single OS's (e.g., Windows services)
  • We will need a centralized way to document these for users to know they are available, without surfing every page in the API.
• Multiple ways to fetch data implemented as "drivers" (e.g., PDH, WMI, command line) that can either be manually or automatically configured as the source for information.
• A common exception type that tells the user there's "no data" or "no new data" (NotFoundException? AccessDeniedException?)

[dblock]

I love the detailed examples here @dbwiddis. ... scratching my head mostly at this point :)

[YoshiEnVerde]

Sorry about the following wall of posts, but I didn't want a single monolothic post that became unreadable

The main problem with not using inheritance is that it defeats pretty much 90% of the point of using OSHI, outside of not having to bother implementing your own calls/parsers for the system values.

Any use case that depends on getting a specific value that can only be recovered in a specific O.S. for a specific architecture, under a specific hardware stack, voids every single point of having a common library.

If you need an ID that can only be gotten in Windows, for a specific windows-only type of service, that can only be gotten in 32bit archs, for a SPARC machine, then you'd be best served just doing a straight call for the service, instead of importing OSHI.
Just copy the corresponding method from OSHI and you'll be done with it.

The idea behind having inheritance is that the 85% of the data that is, relatively, common between all systems should be accessible in three or four lines, without ever even caring what tech stack is under you.

That's also the point of a standardized API, to remove the complexity of all the details, and to abstract the implementation, for the user.

If you need to manually check which OS you're working under, because you have special use-cases for each stack, then you're already doing that check by hand, you can just as well do a manual cast for the returning object, or call a more specialized API (that we would also be providing).

[YoshiEnVerde]

As counter examples:

• I need to check every hour, or so, how much free space I have in my disks. I shouldn't have to care how each OS handles that, I should just call a single method and get a single value back: double System.Memory.Disks.FreeSpace()
• I need to get some serial numbers from the system, I shouldn't care what the exact system is, just call a single method for the values

[YoshiEnVerde]

The point is that we should have multiple layers of interaction:

• Public Facing

  1. Standardized API: Only values that are shared between stacks. Both calls and return types should be standardized for every value, to make it easier to handle interoperability
  2. Stack Specific API: Here we can have all the details and values that are O.S. specific, or Arch specific. These can be called directly, if we have a set stack, or we can use some promotion mechanism to reach them from [1]
  3. Utilities API: These should contain the elements needed to operate on the values from [2], to move from [1] to [2] (or vice-versa), for common utils for [1], etc.

• Internals

  1. Drivers: These are the internals that actually gather the values. Methods and value types for this layer should not reflect on methods and values from [1] or [2]
  2. Cache: This handles caches, staleness, refreshes, etc

[YoshiEnVerde]

Matter of fact is that, currently (OSHI3) there is NO real common interface.

Because we have elements in there that are OS specific (or stack specific), they fail if the OS/Stack is not the expected one, and they basically require the user to already know what they were going to find before calling OSHI, in the end the common interface is just a fallacy.

Even worse, because of that common interface, the values returned are inconsistent across platforms. For example, the issue that originally brought me to OSHI: For one OS GetSerialNumber() returns the OS serial number, for another OS it returns the MOBO serial number, for another it returns the CPU's.