chore: clean code smells

See https://sonarcloud.io/dashboard?id=ReliaQualAssociates_ramstk for overview of smells and duplications that need to be addressed.
Pull Request Checklist

Code Style

[ ] Code is following code style guidelines.
[ ] Followed in new code.
[ ] Corrected in existing code whenever possible.
[ ] Spelling and English grammar errors have been eliminated.
[ ] Attribute and variable names make sense.
[ ] Stub files created or updated.
[ ] New code is readable.
[ ] Execute 'make maintain' before committing changes and fix any issues.
[ ] Code is not overly complicated; refactor if it is.
[ ] Debugging, including print(), statements have been removed.

Tests

[ ] At least one test for all newly created functions/methods?
[ ] Tests capture key use cases.
[ ] Enough edge cases covered for comfort.
[ ] Code coverage has not decreased.

Documentation

[ ] Update api documentation to reflect the changes made.
[ ] Update the user documentation to reflect the changes made.

Chores

[ ] Problem areas outside the scope of this PR have an # ISSUE: comment
decorating the code block.  These # ISSUE: comments are automatically
converted to issues on successful merge.  Alternatively, you can manually
raise an issue for each problem area you identify.
[ ] TODO and FIXME statements have been have been converted to # ISSUE
: comments or removed in their entirety.
[ ] # ISSUE: comments have been removed for those issues this PR
addresses.
[ ] Update the features section of README.md for newly added work stream modules.

All PR checks pass.

[ ] Execute 'make format', 'make stylecheck', 'make typecheck', and 'make
lint' before committing changes and fix any issues.
[ ] Failing static checks are only applicable to code outside the scope of
this PR.

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information
This is a catch-all PR for cleaning up code smells reported by SonarCloud (or other tools/services).

This is a terrible way to address these issues.


Supporting IE

After a slow bout of not-well documented testing, I've answered the following questions involving iframes, different domains, and localStorage. The questions address functionality that a browser platform must provide, in order to support these SubEtha features:

Observe localStorage data & changes.
* Storage events are the message bus, upon which SubEtha stakes it's claim.
Observe iframe unloading.
* Unloading is key to the SubEtha bridge, since it must update the client map (a localStorage key) and notify the network of any departing peers.

Can the iframe access data in same origin?

Safari - no :x:
Opera - yes :heavy_check_mark:
Chrome - yes :heavy_check_mark:
Firefox - yes :heavy_check_mark:
IE 8 - yes :heavy_check_mark:
IE 9 - yes :heavy_check_mark:
IE 10 - yes :heavy_check_mark:
IE 11 - yes :heavy_check_mark:

Safari turned out to be the odd-browser out. Safari's default settings produce a kind of sandboxed storage that isolates iframed pages, even when pointing to the same origin. This behavior is due to user settings, and has no apparent workaround.
Can the iframe receive storage events?

Safari - no :x:
Opera - yes :heavy_check_mark:
Chrome - yes :heavy_check_mark:
Firefox - yes :heavy_check_mark:
IE 8 - yes :heavy_check_mark:
IE 9 - yes :heavy_check_mark:
IE 10 - yes :heavy_check_mark:
IE 11 - no :x:

Safari results are again blamed on the default user setting. All versions of IE require a workaround.
IE 8-10 provide empty storage events with zero relevant details. However, their occurrence is enough to manually track of changes, vs. receiving them in the event.
IE 11 does not provide storage events to different domain iframes. Any solution will require polling.
Does the iframe receive an unload event when the parent navigates?

Safari - yes :heavy_check_mark:
Opera - yes :heavy_check_mark:
Chrome - yes :heavy_check_mark:
Firefox - yes :heavy_check_mark:
IE 8 - yes :heavy_check_mark:
IE 9 - yes :heavy_check_mark:
IE 10 - yes :heavy_check_mark:
IE 11 - yes :heavy_check_mark:

Mercifully, all browsers trigger the unload event when the parent page navigates.
Does the iframe receive an unload event when the node is removed from the parent DOM?

Safari - yes :heavy_check_mark:
Opera - yes :heavy_check_mark:
Chrome - yes :heavy_check_mark:
Firefox - yes :heavy_check_mark:
IE 8 - no :x:
IE 9 - yes :heavy_check_mark:
IE 10 - yes :heavy_check_mark:
IE 11 - yes :heavy_check_mark:

IE 8 simply does not provide this event, when the parent DOM removes the bridge's iframe. Having the parent monitor the DOM is not a solution, since it is the bridge (i.e., iframed document) that must observe the unload event, in order to perform it's clean up routine. This finding effectively eliminates IE 8 as a SubEtha compatible browser.
Next steps?
Given my tests, it's clear that Safari and IE 8 can not support SubEtha. As well, workarounds must be coded for IE support. This bug will track progress in developing the IE workarounds.

Tracking LocalStorage Changes in IE 9 & 10
IE 9 & 10 have empty storage events, which informs when to read storage keys but also allows a race-condition, where the change which triggers the event gets overwritten by the time it is read.
Unique Write Keys
Since writing to a single key can not be coordinated between n number of bridges, the solution must first allow for each bridge to have it’s own writable key. This way each bridge can throttle it’s own writes, and not be concerned about blowing away messages from another bridge.
Registration and Change Manifest
But since the IE solution means bridges no longer share keys, there would be no manifest (or map) listing all known bridges. The race-condition simply prevents using localStorage for syncing data across runtimes. Instead, a more concurrent, shared data store must be employed: cookies.
A single cookie can be shared between bridges (iframes). Upon initializing, each bridge will have a random delay before accessing this cookie, in order to retrieve and update a “bridge manifest”. This could be a simple hash, listing each bridge id. Bridges would need to update this cookie, in order to inform other bridges that they have exited the network.
The bridge manifest would also serve as a change manifest, tracking the last time each bridge updated it’s single storage key. Thus, when a given bridge receives a storage event, it first references the cookie in order to compare timestamps and determine which bridge sourced the event.
Implementation Risks
This solution presumes that cookie access is faster than localStorage. It also presumes that localStorage reads and writes are either blocking or immediately shared between window instances. That is, how soon is a set storage value available to other iframes?
cc: @cnojima

A Cookie I/O test
To test how long it takes for an updated cookie value to be available to other windows, I created two jsbin test pages. The first increments a cookie value every given number of milliseconds (20, by default). The second reads the value of this cookie every given number of milliseconds (5, by default).
With a new cookie value being set every 20ms in one window, and read every 5ms in another, the hope is to view a log of sequentially incrementing values. That is, the log reads out a few lines of "1" (since the cookie won't change until 20ms later), then lines of "2", and lines of "3". Lines that don't have a sequential value, are suffixed with an asterix. The results of this test will be posted here, over time. They should give a clearer picture about the latency to be considered, in the IE implementation.

The cookie-manifest approach attempts to reduce the i/o needed to observe changes in localStorage. Below is an illustration of the expected sequence. In brief, setting and getting localStorage values will involve setting and reading a shared cookie, first.

Based on earlier tests, new storage and cookie values would be available when the corresponding storage event occurs. Combined with throttling writes by ~40ms, the architecture should ensure messages are not dropped. As well, this avoids having to store previous messages or unnecessarily read storage-keys (i.e., the cookie will inform which keys have been updated).
IE 11 can use this architecture, except it's trigger for reading storage-keys would be changes in the cookie-manifest - not the storage event. This is because IE11 doesn't support storage events for iframes in different domains (some forking may be done, to support the best change monitor/management approach).
The cookie-manifest does rely on reading and writing to a cookie that will be shared by n number of iframes. This is understood to be a risk, but is the only idea I've found that avoids any kind of heartbeat style monitoring. The latter requires each bridge track the presence of other bridges, using timestamps. It may be worth looking into, but definitely requires more overhead and guarantees more latency... will look into after testing the cookie-manifest approach.

Making definite progress with IE implementation. As suspected, the unload logic is either (1) not firing, or (2) not cleaning up the cookie and storage keys as needed.
I'm trying to use an cleanup routines for "expired" bridges - bridges that are not in the manifest, but have a tracking message. It will make everything rather slow, but necessary to avoid "ghost" peers that hang around per client.

Works in IE11 too. Gonna clean up for IE9 next. Final test is to see if the winfo visualizer works as expected.

There is an issue with the visualizer, where either clients or client events are dropped. But, I fear it's in that code, not due to the bridge itself. I won't close this until the visualizer works as expected in IE browsers.

The IE manifest solution began to have race-condition failures from too many browser windows attempting to read and write to one cookie. A heartbeat approach will ensure bridges can track when other bridges drop and join the network.
The approach calls for each bridge to update a tracked cookie value, every X number of milliseconds. Other bridges will check the cookie every X/2 milliseconds, to ensure each bridge is captured and removed as needed.
The heartbeat addition is rather minor, but the working code has many moving parts and will still take some time to implement. Below is the latest sequence diagram, demonstrating more of the full network actions and logic behind SubEtha so far.

Eliminating dropped messages
Just chatted with @SerpentJoe, who came up with a brilliant method for ensuring zero dropped messages in IE: self-receipts. The idea is that the message manifest would serve as both a signal for new messages (to other bridges) and a confirmation that messages have been "received" (for the same bridge).
After bridge A updates it's message ticker (in the message manifest/cookie), it's next check of the cookie would confirm that the changed value is present. If not, the cookie is again set, to the last ticker value. This handles the case where a cookie write may be blown away by another bridge that happened to be set the cookie at the same time.
The only case where this doesn't work, is when the last message is dropped, before the bridge exits. It requires bridges to write near each other, but that should be generally avoidable.
There's more to this idea, but it adds a great deal of confidence to this headache laden attempt to get SubEtha to work on IE. Thanks again, to Peter Jones, and I'll post another sequence diagram in a few days.

Ok, this looks like it should work. This sequence diagram may be updated in the future, but it best captures the "self-receipt" concept, inspired by Peter Jones (@serpentjoe). I'll be coding this for the next week or so, in order to eliminate dropped messages and ghost peers.

Okay. I've clearly been discouraged by this polling/cookie/localStorage approach, up till now required for SubEtha to work in IE. Forking the implementation between it and standards compliant-er browsers simply blows chunks. It hurt my brain trying figure out a timing strategy, based on the number of active bridges. (I'm terrible with numbers.)
But then, @gothy takes note of this project and stars it. I was inspired to push onwards, presuming folks want a working idea, not just a good one. Turns out, he left a comment on crosstab (a similarly purposed library), issue tejacques/crosstab#26 ... Not surprisingly that project has equally difficult localStorage challenges in IE.
The Comment that Could
His comments links to a localStorage bug against IE 11. The bug lists some workarounds, the last of which recommends using indexedDB (IDB).
I had heard of this API, and confused it for WebSQL, a narrowly supported client-storage feature. IDB on the other hand, has wider browser support. Even better news: preliminary tests indicate how IndexedDB will become a different domain iframes in IE 10.
That said, put simply, I disliked their API. It took a day for me to nail down the choreography of objects and methods needed for minimal usage. Nonetheless, with my newfound understanding, I'm excited to code the SubEtha-Bridge module using IDB exclusively - removing any browser forks.
IndexedDB Strategy
The manifest/msg concept will be preserved for IDB.  The main difference is that IDB will not dispose of keys - the localStorage approach reuses the same (designated) key. The former means I can use IDB as a message log!
IDB is not an event-based system, such as localStorage. Thus, the first pill to swallow is that IDB will require polling. The good news is that the each poll has a response,  making the practice less blind and computationally intensive (a la setInterval).

IndexedDB Strategy (Part 2)
Looks like IndexedDB handles all the concurrency issues introduced by the peerless'ish network I'm attempting to create with SubEtha. It's the centralized logic I was always missing!
So what's the bad news? FireFox and Safari don't allow IndexedDB within different domain iframes. That Safari config may allow it, but there's no corresponding config in FF. This is a true bummer, but I'll take Chrome, IE 10, and IE 11, over Safari and Firefox.
The best news is that the architecture will be much cleaner, codified into IndexedDB stores ("tables" in SQL parlance). The database schema will actually offload much of the logic currently handled by each bridge. For example, IndexedDB will obviate the need for network messages, since each bridge can access the same record of changes.
The Schema
These are the tables and columns I envision thus far. (It's been some time since I've used SQL lingo, so forgive my rough edges.)

Table "bridge"

id - A generated unique identifier. (number, auto-increment, primary)
url - The bridge url. (string)
ping - The last time a bridge updated this column. (date)

Table "client"

id - A unique identifier, given by the client. (string, primary)
bid - The identifier of the bridge hosting this client. (number)
url - The url of the client, when it connected. (string)
created - The date the client connected. (date)

Table "message"

id - A unique identifier, given by the client. (string, primary)
sender - The identifier of the sending client. (string, primary)
receiver - The identifier(s) of the receiving client(s). (string/array)
type - The message type, indicating how it should be processed. (string)

content - The message content. (mixed)

Glad to hear that starring this repo was inspirational :)

Below is the new sequence diagram for SubEtha, using IndexedDB. It mentions Promises, since the SubEtha-Client will be undergoing changes to support them (see bemson/subetha-client#3).

Unique IDs and security
This new architecture will rely on IDB for unique ids. That is, both bridges and clients will get their unique ids from the ones auto-generated by IDB. This should save a few bytes and processing, since no guids are required - the ids no longer need to be random, since they are canonical to the IDB object store indice.
However, in order to prevent unsanctioned messages, where one client fakes it's id, a local index must also exist. The local index will simply increment ids of SubEtha clients in the browser. The bridge (a sandboxed execution environment) will map the local id to the "global" id - for lack of a better term.
Thus, clients must use the same local id when sending event, or the bridge will reject it and sever the client connection.

This is very interesting! Please let me know if you end up getting this working, and I may adopt this or a similar strategy to deal with IE10/11 iframes for crosstab.

Thanks very much for the interest @tejacques !
My efforts are split between trying to plough through the code and keeping perspective on the approach. I really feel that if I could better document the "protocol" I'm trying to codify, the simpler it'd be to code it and get peer feedback.
Below are the basic payloads I envision for this IDB variation.
I'm working on publishing this to the wiki, but docs are not my strong suit. Also, the wiki would provide a roadmap that puts standards-based encryption on the schedule - which will become more and more important to devs.

SubEtha Message Payloads / DRAFT
This is the minimum message structure required by the bridge (for the IDB strategy).
{
    mid: <int>,         // message identifier
    type: <string>      // message type
}

From the host or bridge, requirements for the remaining message structure would depend on the following message types.

request (host -> bridge) - the host is asking something regarding the network

join - host requests a new client connection
event - host requests sending a client event

response (bridge -> host) - response to either a join or event request
drop (host <-> bridge) - host or bridge instructs to drop a client
die (bridge -> host) - bridge informs host it is closing (...spec for completeness?)

(Host) Request Payload
Handlers for join or event requests, would require an additional request identifier. The bridge response would use this identifier in order to link it's message to the request.
{
    mid: <int>,        // message identifier
    type: <string>,    // message type
    data: {            // message payload
        rid: <int>         // request identifier
    }
}

Note: The wiki will go into details for both request types.
(Bridge) Response Payload
A bridge response adds to the host's request structure. To link a response to a request, it would reference the same identifier given by the host.
The "status" field allows for sending HTTP-style response code, so clients could glean more details about the result (specifically in cases of failure). I have yet to list these codes, but they're occurring as I find exit points in the code (e.g., while parsing an incoming message).
{
    mid: <int>,        // message identifier
    type: 'rsp',       // message type
    data: {            // message payload
        rid: <int>,        // request identifier
        ok: <bool>,        // request result
        status: <int>      // response status
    }
}

Notes

This spec omits the transport wrappers implemented by both the bridge and host when transmitting messages. This will likely not be part of any spec, but instead be implementation dependent.
This "protocol" has yet to describe the initial handshake between the host and bridge, which establishes a secure connection for transmitting the messages.

Okay - not the update I'd hoped to post, but I am making progress in planning the work for this bridge refactor. I'm making changes, but have yet to publish my branch since the code isn't in "commitable" shape, right now.
So, despite having sequence-diagram-fatigue, below is my latest understanding of how this architecture will play out with IndexedDB. Of note, I've added details on cancelling a join request - previously called "auth" request. There are some other linguistic changes, like calling client events client messages. But, the basic flows are the same.

What I would like to do soon (besides progress in actually coding this), is reconcile these flows with the various message types.

The Tao of LocalStorage and IE
It has been an uncomfortably long while since updating this thread. After so much support and interest from peers, it felt prudent to capture something now, before addressing the next challenge. So, forgive me in advance for this laborious review and reflection of the past ten odd months of frustratingly slow progress.
The Demo: Counting Together
As one would hope of any self-respecting technology article, there is a SubEtha Bridge demonstration. The demo is very rudimentary: rendering but a single number on the page. That number reflects how many other bridges are on the veritable localStorage "network"... Riveting stuff, isn't it?
To see it in action, load the url in multiple windows (tabs, are fine, but far less interesting). As you open and close them, each window will increment and decrement it's number. Latency aside, the synchronized updating of separate windows shows how they are in fact linked.
Here is a short screen capture of the demo, using four IE 10 windows.

Technical Errata
Though it uses my Bridge module, the demo is meant to example how localStorage can be used in IE, for inter-window communications. The demo code actually co-opts a specific version of the Bridge module, and fools it to work outside an iframe and with a mock Client module.
Because this demo is not in an iframe, it should work in Firefox and Safari browsers. However, the larger SubEtha project will continue to target Chrome, IE 10, and IE 11 only. (Respectively, by policy and default, Firefox and Safari do not allow storage access in third-party iframes.)
IE Edge was the outlier for this solution. The approach may work with non-iframed, same-origin windows in IE Edge. Unfortunately, for my purposes, IE Edge is a lost cause...
The Problem: Uneventful Events
Storage events offer the ability for separate browser instances to observe changes to their shared, origin-based localStorage items. The event would describe when and how a given item (or "key") changed. Withstanding Firefox and Safari's paranoid security policies, IE is the only browser to get this wrong.
IE's infamous, abysmal, storage event implementation provides zero information on the nature of a storage change. At best, IE storage events inform you that a change occurred. The result is out of sync logic and an impossibly disconnected "network".
The Solution: Treat LocalStorage Like a Hash
Without useful storage events in IE, localStorage should be treated as the mundane (albeit persisted) hash object it always was. Normal best-practices can be applied, as if you were dealing with any old shared object. Also, considering that this hash will be touched by several untrusted routines, you might then imagine measures to monitor changes made by other routines.
Once it became clear that I could never artificially broadcast localStorage changes to other windows, figuring out a way to communicate became more straightforward (less "feature magic" from the browser). I used key additions, updates and removals, to send messages between windows.
Proper Namespacing Applies
In IE, for any routine that uses localStorage, you should use standard namespacing. The difference is that each namespace would need to be wholly unique, in order to avoid sharing keys (akin to namespace collisions). In my module, bridges generated a unique/random id, and prefixed their storage items with that id.
By not sharing keys, you avoid data racing. A data race is similar to a race condition, except it's the data which is not concurrent (not the execution). For instance, multiple windows writing to the same key will leads to data loss, since each window can only read the latest value.

The magic of storage events ameliorated these data race/loss scenarios. Logic no longer relied on the current value, but transitional one provided by the storage event.

Monitor Changes Manually
With localStorage - an ever changing shared hash - you will need to observe object member changes yourself. However you implement it (i.e., the events you define and their payloads), this boils down to periodically reconciling keys with some cached value.
That effort led to a monitoring routine and some custom events. My work was tweaked for my module, but it could be extracted as a separate module.
Monitoring any object this way is absolutely non-performant. But it does offer the next best thing to native "storage events". The primary difference being that you can only catch some changes with this method - storage events (are meant to) capture every transition of a storage item.
So, to measure bad and good news, this is not a way to monitor fast changing storage items. Nor does it guarantee that a changed or removed item will be observed at all. More so, the term "fast" is relative, and complicated by browser-specific i/o details that I loathe to dive into here. (Not that I know about them, mind you - just that low-level details always have a long tail.)
Ultimately, in order for this error prone approach to work, you'll need to adjust how you use localStorage entirely...
Use the Hash Like a Log
With a poor-mans object monitor, it's best to change values as little as possible. So instead of capturing an object in localStorage, capture changes to the object's state. Said another way, you want to capture your custom storage events (that you don't receive in IE) in localStorage.
For example, here's how you might trace the on/off state of a light-bulb over time. Instead of making a key reflect the bulb's current state, you would add keys which reflect changes to that bulbs state.

Key
Value

light-<guid>
{"watts": "60", "room": "den", "created": 12310920}

light-<guid>-change-1
{"on":1,"ts":12345678}

light-<guid>-change-2
{"on":0,"ts":12346678}

light-<guid>-change-3
{"on":1,"ts":12347678}

light-<guid>-change-4
{"on":0,"ts":12348678}

By designating a key-naming pattern that identifies the bulb's static characteristics, you can effectively communicate it's dynamic characteristics to other windows. In this example, changes to the on/off state are sequenced in the key name. The actual value further communicates the timing of the state change.
With this approach, any slower performing windows can catch up to every transition you expected with normal storage events.
Sort and Prioritize Custom Notifications
Regarding key-naming patterns, it's important to use a scheme that prioritizes the messages you want to communicate between two windows. Continuing with the bulb example, a bulb that breaks, might use the key break-<guid>, where the actual "guid" matches a key named light-<guid>. Before or after reconciling which keys changed, you would then process broken bulb events before bulb change events. The importance of these details depend on your goals.
Devise a Protocol for Value Changes and Key Removal
This approach doesn't mean never remove a key. But it does mean that you want to design how your system should respond when certain keys are removed. Some keys won't matter - like the bulb state change keys. Some would - like the bulb identity keys.
In my case, I used the removal of a value to communicate meaning to other windows. My particular IndexedDB cleanup process required the aid of the remaining Bridges.
When a window closes, the Bridge's identity key is set to "0". Other bridges see this, then line up to determine who will clean up the expired bridge. Once one of those bridges finish, it removes the identity key of the expired bridge. The removal of the key informs the bridges that the deed is done.
In this way, I update a bridge's value once to communicate a binary state change: alive or dead. Then I use the removal of the key to further communicate to bridges that the expired bridge has been cleaned up.
IE LocalStorage Gotchas
While this strategy for addressing IE's localStorage shortcomings works, there are a few surprises that I wanted to relay as well.
Mind the Gap
IE 10 and IE 11 are completely different, when it comes to localStorage events: One has them, the other doesn't. My poor shim had to use different techniques for assessing data changes.
With IE 10, I let the events trigger scans. However, since each write results in several events, I chose to debounce them a bit. I will likely throttle them in the future, to avoid having perpetual events preventing otherwise timely scans.
In IE 11, there are no events to speak of, so I simply scan localStorage at a reasonable interval. Because scans are costly, I plan to make that interval dynamic, based on the frequency of changes encountered.
Calling all Keys
I must warn against these two methods of reading localStorage keys in IE: Object.keys(localStorage), and for (key in localStorage) {...}. Using either will likely result in missing keys that have been added by other windows.
The safest way to retrieve all localStorage keys/values, is to use the .length property along with the .key() and .getItem() methods.
var
  ln = localStorage.length,
  cache = {},
  key
;

while (ln--) {
  key = localStorage.key(ln);
  cache[key] = localStorage.getItem(key);
}

Raise Your Hand if You're Not Here
Even with the recommended approach for reading data, I've been able to retrieve a key who's value is null.
So don't assume that a key's existing means it is valid. A simple string-type check should separate the dead from the living. This will avoid emitting an update-like event on keys that have actually removed.
Drop Kick IE Often
The most surprising discovery came from observing that some windows simply stopped observing localStorage changes. Like forcing a CSS redraw, localStorage must also be kicked in to gear, so it's data stays in sync.
Though an expensive operation, syncing was mercifully simple: just write something to localStorage. Thankfully, so far, I've only observed this in IE 10.
function ieKickStorage() {
  localStorage.setItem('somekey', 1);
}

Based on my experiments, neither the key nor value has to change between writes - which is somewhat counterintuitive. I'm unsure if removing a key would have the same effect.
In my Bridge module, I "kick" localStorage every 3 seconds with the same key and value. I plan to add some finesse this, like resetting my interval after actually writing data.
What's Next?
Firstly, this issue is not solved in my opinion. However, my progress does allow moving on to other topics. The lack of IE Edge support will likely be addressed separately as a new bug. (Hasn't this one been through enough?)
Secondly, I'd like to put time towards a demo that doesn't include my Bridge module. It's really a distraction from what was learned here. By extracting my localStorage monitor, one could easily write logic and a key structure that monitors the addition and removal of windows... Any takers?
What About SubEtha?
I grew fond of the diagrams I created in this thread, and have updated them - but they mainly describe SubEtha, not the Bridge. I hope to post them in the SubEtha wiki (which is empty now).
I also need to update the Client module to work with the new Bridge module. While these tasks require time, neither should take too long.
Gratitude & Acknowledgments
Thanks to the following individuals for seeing me through this insane issue. At some critical moment, these folks offered their time and attention, if not enthusiasm and interest in this issue. For that, I thank you - in order of your user name:

@aversini
@bossroller
@cnojima
@davidrekow
@gothy
@puzrin
@reptile
@ryangiomi
@serpentjoe
@tejacques

Great write up, and congrats!  Had a laugh about IE and cowballs.  :D

From: Bemi<EMAIL_ADDRESS>Sent: 10/29/2015 20:27
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS>Subject: Re: [subetha-bridge] Supporting IE (#1)
The Tao of LocalStorage and IE
It has been an uncomfortably long while since updating this thread. After so much support and interest from peers, it felt prudent to capture something now, before addressing the next challenge. So, forgive me in advance for this laborious review and reflection of the past ten odd months of frustratingly slow progress.
The Demo: Counting Together
As one would hope of any self-respecting technology article, there is a SubEtha Bridge demonstration. The demo is very rudimentary: rendering but a single number on the page. That number reflects how many other bridges are on the veritable localStorage "network"... Riveting stuff, isn't it?
To see it in action, load the url in multiple windows (tabs, are fine, but far less interesting). As you open and close them, each window will increment and decrement it's number. Latency aside, the synchronized updating of separate windows shows how they are in fact linked.
Here is a short screen capture of the demo, using four IE 10 windows.

Technical Errata
Though it uses my Bridge module, the demo is meant to example how localStorage can be used in IE, for inter-window communications. The demo code actually co-opts a specific version of the Bridge module, and fools it to work outside an iframe and with a mock Client module.
Because this demo is not in an iframe, it should work in Firefox and Safari browsers. However, the larger SubEtha project will continue to target Chrome, IE 10, and IE 11 only. (Respectively, by policy and default, Firefox and Safari do not allow storage access in third-party iframes.)
IE Edge was the outlier for this solution. The approach may work with non-iframed, same-origin windows in IE Edge. Unfortunately, for my purposes, IE Edge is a lost cause...
The Problem: Uneventful Events
Storage events offer the ability for separate browser instances to observe changes to their shared, origin-based localStorage items. The event would describe when and how a given item (or "key") changed. Withstanding Firefox and Safari's paranoid security policies, IE is the only browser to get this wrong.
IE's infamous, abysmal, storage event implementation provides zero information on the nature of a storage change. At best, IE storage events inform you that a change occurred. The result is out of sync logic and an impossibly disconnected "network".
The Solution: Treat LocalStorage Like a Hash
Without useful storage events in IE, localStorage should be treated as the mundane (albeit persisted) hash object it always was. Normal best-practices can be applied, as if you were dealing with any old shared object. Also, considering that this hash will be touched by several untrusted routines, you might then imagine measures to monitor changes made by other routines.
Once it became clear that I could never artificially broadcast localStorage changes to other windows, figuring out a way to communicate became more straightforward (less "feature magic" from the browser). I used key additions, updates and removals, to send messages between windows.
Proper Namespacing Applies
In IE, for any routine that uses localStorage, you should use standard namespacing. The difference is that each namespace would need to be wholly unique, in order to avoid sharing keys (akin to namespace collisions). In my module, bridges generated a unique/random id, and prefixed their storage items with that id.
By not sharing keys, you avoid data racing. A data race is similar to a race condition, except it's the data which is not concurrent (not the execution). For instance, multiple windows writing to the same key will leads to data loss, since each window can only read the latest value.

The magic of storage events ameliorated these data race/loss scenarios. Logic no longer relied on the current value, but transitional one provided by the storage event.

Monitor Changes Manually
With localStorage - an ever changing shared hash - you will need to observe object member changes yourself. However you implement it (i.e., the events you define and their payloads), this boils down to periodically reconciling keys with some cached value.
That effort led to a monitoring routine and some custom events. My work was tweaked for my module, but it could be extracted as a separate module.
Monitoring any object this way is absolutely non-performant. But it does offer the next best thing to native "storage events". The primary difference being that you can only catch some changes with this method - storage events (are meant to) capture every transition of a storage item.
So, to measure bad and good news, this is not a way to monitor fast changing storage items. Nor does it guarantee that a changed or removed item will be observed at all. More so, the term "fast" is relative, and complicated by browser-specific i/o details that I loathe to dive into here. (Not that I know about them, mind you - just that low-level details always have a long tail.)
Ultimately, in order for this error prone approach to work, you'll need to adjust how you use localStorage entirely...
Use the Hash Like a Log
With a poor-mans object monitor, it's best to change values as little as possible. So instead of capturing an object in localStorage, capture changes to the object's state. Said another way, you want to capture your custom storage events (that you don't receive in IE) in localStorage.
For example, here's how you might trace the on/off state of a light-bulb over time. Instead of making a key reflect the bulb's current state, you would add keys which reflect changes to that bulbs state.

Key
Value

light-<guid>
{"watts": "60", "room": "den", "created": 12310920}

light-<guid>-change-1
{"on":1,"ts":12345678}

light-<guid>-change-2
{"on":0,"ts":12346678}

light-<guid>-change-3
{"on":1,"ts":12347678}

light-<guid>-change-4
{"on":0,"ts":12348678}

By designating a key-naming pattern that identifies the bulb's static characteristics, you can effectively communicate it's dynamic characteristics to other windows. In this example, changes to the on/off state are sequenced in the key name. The actual value further communicates the timing of the state change.
With this approach, any slower performing windows can catch up to every transition you expected with normal storage events.
Sort and Prioritize Custom Notifications
Regarding key-naming patterns, it's important to use a scheme that prioritizes the messages you want to communicate between two windows. Continuing with the bulb example, a bulb that breaks, might use the key break-<guid>, where the actual "guid" matches a key named light-<guid>. Before or after reconciling which keys changed, you would then process broken bulb events before bulb change events. The importance of these details depend on your goals.
Devise a Protocol for Value Changes and Key Removal
This approach doesn't mean never remove a key. But it does mean that you want to design how your system should respond when certain keys are removed. Some keys won't matter - like the bulb state change keys. Some would - like the bulb identity keys.
In my case, I used the removal of a value to communicate meaning to other windows. My particular IndexedDB cleanup process required the aid of the remaining Bridges.
When a window closes, the Bridge's identity key is set to "0". Other bridges see this, then line up to determine who will clean up the expired bridge. Once one of those bridges finish, it removes the identity key of the expired bridge. The removal of the key informs the bridges that the deed is done.
In this way, I update a bridge's value once to communicate a binary state change: alive or dead. Then I use the removal of the key to further communicate to bridges that the expired bridge has been cleaned up.
IE LocalStorage Gotchas
While this strategy for addressing IE's localStorage shortcomings works, there are a few surprises that I wanted to relay as well.
Mind the Gap
IE 10 and IE 11 are completely different, when it comes to localStorage events: One has them, the other doesn't. My poor shim had to use different techniques for assessing data changes.
With IE 10, I let the events trigger scans. However, since each write results in several events, I chose to debounce them a bit. I will likely throttle them in the future, to avoid having perpetual events preventing otherwise timely scans.
In IE 11, there are no events to speak of, so I simply scan localStorage at a reasonable interval. Because scans are costly, I plan to make that interval dynamic, based on the frequency of changes encountered.
Calling all Keys
I must warn against these two methods of reading localStorage keys in IE: Object.keys(localStorage), and for (key in localStorage) {...}. Using either will likely result in missing keys that have been added by other windows.
The safest way to retrieve all localStorage keys/values, is to use the .length property along with the .key() and .getItem() methods.
var
  ln = localStorage.length,
  cache = {},
  key
;

while (ln--) {
  key = localStorage.key(ln);
  cache[key] = localStorage.getItem(key);
}

Raise Your Hand if You're Not Here
Even with the recommended approach for reading data, I've been able to retrieve a key who's value is null.
So don't assume that a key's existing means it is valid. A simple string-type check should separate the dead from the living. This will avoid emitting an update-like event on keys that have actually removed.
Drop Kick IE Often
The most surprising discovery came from observing that some windows simply stopped observing localStorage changes. Like forcing a CSS redraw, localStorage must also be kicked in to gear, so it's data stays in sync.
Though an expensive operation, syncing was mercifully simple: just write something to localStorage. Thankfully, so far, I've only observed this in IE 10.
function ieKickStorage() {
  localStorage.setItem('somekey', 1);
}

Based on my experiments, neither the key nor value has to change between writes - which is somewhat counterintuitive. I'm unsure if removing a key would have the same effect.
In my Bridge module, I "kick" localStorage every 3 seconds with the same key and value. I plan to add some finesse this, like resetting my interval after actually writing data.
What's Next?
Firstly, this issue is not solved in my opinion. However, my progress does allow moving on to other topics. The lack of IE Edge support will likely be addressed separately as a new bug. (Hasn't this one been through enough?)
Secondly, I'd like to put time towards a demo that doesn't include my Bridge module. It's really a distraction from what was learned here. By extracting my localStorage monitor, one could easily write logic and a key structure that monitors the addition and removal of windows... Any takers?
What About SubEtha?
I grew fond of the diagrams I created in this thread, and have updated them - but they mainly describe SubEtha, not the Bridge. I hope to post them in the SubEtha wiki (which is empty now).
I also need to update the Client module to work with the new Bridge module. While these tasks require time, neither should take too long.
Gratitude & Acknowledgments
Thanks to the following individuals for seeing me through this insane issue. At some critical moment, these folks offered their time and attention, if not enthusiasm and interest in this issue. For that, I thank you - in order of your user name:

@aversini
@bossroller
@cnojima
@davidrekow
@gothy
@puzrin
@reptile
@ryangiomi
@serpentjoe
@tejacques

Reply to this email directly or view it on GitHub:
https://github.com/bemson/subetha-bridge/issues/1#issuecomment-152403634

Thanks mayne. More to come! Enjoy the Halloween weekend!
On 10/29/15, ryangiomi<EMAIL_ADDRESS>wrote:

Great write up, and congrats!  Had a laugh about IE and cowballs.  :D

From: Bemi<EMAIL_ADDRESS>Sent: 10/29/2015 20:27
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS>Subject: Re: [subetha-bridge] Supporting IE (#1)
The Tao of LocalStorage and IE
It has been an uncomfortably long while since updating this thread. After so
much support and interest from peers, it felt prudent to capture something
now, before addressing the next challenge. So, forgive me in advance for
this laborious review and reflection of the past ten odd months of
frustratingly slow progress.
The Demo: Counting Together
As one would hope of any self-respecting technology article, there is a
SubEtha Bridge demonstration. The demo is very
rudimentary: rendering but a single number on the page. That number reflects
how many other bridges are on the veritable localStorage "network"...
Riveting stuff, isn't it?
To see it in action, load the url in multiple windows (tabs, are fine, but
far less interesting). As you open and close them, each window will
increment and decrement it's number. Latency aside, the synchronized
updating of separate windows shows how they are in fact linked.
Here is a short screen capture of the demo, using four IE 10 windows.

Technical Errata
Though it uses my Bridge module, the demo is meant to example how
localStorage can be used in IE, for inter-window communications. The demo
code actually co-opts a specific version of the Bridge
module,
and fools it to work outside an iframe and with a mock Client
module.
Because this demo is not in an iframe, it should work in Firefox and
Safari browsers. However, the larger SubEtha project will
continue to target Chrome, IE 10, and IE 11 only. (Respectively, by policy
and default, Firefox and Safari do not allow storage access in third-party
iframes.)
IE Edge was the outlier for this solution. The approach may work with
non-iframed, same-origin windows in IE Edge. Unfortunately, for my purposes,
IE Edge is a lost cause...
The Problem: Uneventful Events
Storage events offer the ability for
separate browser instances to observe changes to their shared, origin-based
localStorage items. The event would describe when and how a given item (or
"key") changed. Withstanding Firefox and Safari's paranoid security
policies, IE is the only browser to get this wrong.
IE's
infamous,
abysmal,
storage event
implementation
provides zero information on the nature of a storage change. At best, IE
storage events inform you that a change occurred. The result is out of sync
logic and an impossibly disconnected "network".
The Solution: Treat LocalStorage Like a Hash
Without useful storage events in IE, localStorage should be treated as the
mundane (albeit persisted) hash object it always was. Normal best-practices
can be applied, as if you were dealing with any old shared object. Also,
considering that this hash will be touched by several untrusted routines,
you might then imagine measures to monitor changes made by other routines.
Once it became clear that I could never artificially broadcast localStorage
changes to other windows, figuring out a way to communicate became more
straightforward (less "feature magic" from the browser). I used key
additions, updates and removals, to send messages between windows.
Proper Namespacing Applies
In IE, for any routine that uses localStorage, you should use standard
namespacing. The difference is that each namespace would need to be wholly
unique, in order to avoid sharing keys (akin to namespace collisions). In my
module, bridges generated a unique/random id, and prefixed their storage
items with that id.
By not sharing keys, you avoid data racing. A data
race is similar to a race
condition, except it's the
data which is not concurrent (not the execution). For instance, multiple
windows writing to the same key will leads to data loss, since each window
can only read the latest value.

The magic of storage events ameliorated these data race/loss scenarios.
Logic no longer relied on the current value, but transitional one provided
by the storage event.

Monitor Changes Manually
With localStorage - an ever changing shared hash - you will need to observe
object member changes yourself. However you implement it (i.e., the events
you define and their payloads), this boils down to periodically reconciling
keys with some cached value.
That effort led to a monitoring routine and some custom events. My work was
tweaked for my module, but it could be extracted as a separate
module.
Monitoring any object this way is absolutely non-performant. But it does
offer the next best thing to native "storage events". The primary difference
being that you can only catch some changes with this method - storage
events (are meant to) capture every transition of a storage item.
So, to measure bad and good news, this is not a way to monitor fast
changing storage items. Nor does it guarantee that a changed or removed item
will be observed at all. More so, the term "fast" is relative, and
complicated by browser-specific i/o details that I loathe to dive into here.
(Not that I know about them, mind you - just that low-level details always
have a long tail.)
Ultimately, in order for this error prone approach to work, you'll need to
adjust how you use localStorage entirely...
Use the Hash Like a Log
With a poor-mans object monitor, it's best to change values as little as
possible. So instead of capturing an object in localStorage, capture changes
to the object's state. Said another way, you want to capture your custom
storage events (that you don't receive in IE) in localStorage.
For example, here's how you might trace the on/off state of a light-bulb
over time. Instead of making a key reflect the bulb's current state, you
would add keys which reflect changes to that bulbs state.

Key
Value

light-<guid>
{"watts": "60", "room": "den", "created": 12310920}

light-<guid>-change-1
{"on":1,"ts":12345678}

light-<guid>-change-2
{"on":0,"ts":12346678}

light-<guid>-change-3
{"on":1,"ts":12347678}

light-<guid>-change-4
{"on":0,"ts":12348678}

By designating a key-naming pattern that identifies the bulb's static
characteristics, you can effectively communicate it's dynamic
characteristics to other windows. In this example, changes to the on/off
state are sequenced in the key name. The actual value further communicates
the timing of the state change.
With this approach, any slower performing windows can catch up to every
transition you expected with normal storage events.
Sort and Prioritize Custom Notifications
Regarding key-naming patterns, it's important to use a scheme that
prioritizes the messages you want to communicate between two windows.
Continuing with the bulb example, a bulb that breaks, might use the key
break-<guid>, where the actual "guid" matches a key named light-<guid>.
Before or after reconciling which keys changed, you would then process
broken bulb events before bulb change events. The importance of these
details depend on your goals.
Devise a Protocol for Value Changes and Key Removal
This approach doesn't mean never remove a key. But it does mean that you
want to design how your system should respond when certain keys are removed.
Some keys won't matter - like the bulb state change keys. Some would - like
the bulb identity keys.
In my case, I used the removal of a value to communicate meaning to other
windows. My particular IndexedDB cleanup process required the aid of the
remaining Bridges.
When a window closes, the Bridge's identity key is set to "0". Other bridges
see this, then line up to determine who will clean up the expired bridge.
Once one of those bridges finish, it removes the identity key of the expired
bridge. The removal of the key informs the bridges that the deed is done.
In this way, I update a bridge's value once to communicate a binary state
change: alive or dead. Then I use the removal of the key to further
communicate to bridges that the expired bridge has been cleaned up.
IE LocalStorage Gotchas
While this strategy for addressing IE's localStorage shortcomings works,
there are a few surprises that I wanted to relay as well.
Mind the Gap
IE 10 and IE 11 are completely different, when it comes to localStorage
events: One has them, the other doesn't. My poor shim had to use different
techniques for assessing data changes.
With IE 10, I let the events trigger scans. However, since each write
results in several events, I chose to
debounce them a bit.
I will likely
throttle them in the
future, to avoid having perpetual events preventing otherwise timely scans.
In IE 11, there are no events to speak of, so I simply scan localStorage at
a reasonable interval. Because scans are costly, I plan to make that
interval dynamic, based on the frequency of changes encountered.
Calling all Keys
I must warn against these two methods of reading localStorage keys in IE:
Object.keys(localStorage), and for (key in localStorage) {...}. Using
either will likely result in missing keys that have been added by other
windows.
The safest way to retrieve all localStorage keys/values, is to use the
.length property along with the .key() and .getItem() methods.
var
  ln = localStorage.length,
  cache = {},
  key
;

while (ln--) {
  key = localStorage.key(ln);
  cache[key] = localStorage.getItem(key);
}

Raise Your Hand if You're Not Here
Even with the recommended approach for reading data, I've been able to
retrieve a key who's value is null.
So don't assume that a key's existing means it is valid. A simple
string-type check should separate the dead from the living. This will avoid
emitting an update-like event on keys that have actually removed.
Drop Kick IE Often
The most surprising discovery came from observing that some windows simply
stopped observing localStorage changes. Like forcing a CSS redraw,
localStorage must also be kicked in to gear, so it's data stays in sync.
Though an expensive operation, syncing was mercifully simple: just write
something to localStorage. Thankfully, so far, I've only observed this in IE
10.
function ieKickStorage() {
  localStorage.setItem('somekey', 1);
}

Based on my experiments, neither the key nor value has to change between
writes - which is somewhat counterintuitive. I'm unsure if removing a key
would have the same effect.
In my Bridge module, I "kick" localStorage every 3 seconds with the same key
and value. I plan to add some finesse this, like resetting my interval after
actually writing data.
What's Next?
Firstly, this issue is not solved in my opinion. However, my progress does
allow moving on to other topics. The lack of IE Edge support will likely be
addressed separately as a new bug. (Hasn't this one been through enough?)
Secondly, I'd like to put time towards a demo that doesn't include my Bridge
module. It's really a distraction from what was learned here. By extracting
my localStorage monitor, one could easily write logic and a key structure
that monitors the addition and removal of windows... Any takers?
What About SubEtha?
I grew fond of the diagrams I created in this thread, and have updated them

but they mainly describe SubEtha, not the Bridge. I hope to post them in
the SubEtha wiki (which is empty
now).

I also need to update the Client module to work
with the new Bridge module. While these tasks require time, neither should
take too long.
Gratitude & Acknowledgments
Thanks to the following individuals for seeing me through this insane issue.
At some critical moment, these folks offered their time and attention, if
not enthusiasm and interest in this issue. For that, I thank you - in order
of your user name:

@aversini
@bossroller
@cnojima
@davidrekow
@gothy
@puzrin
@reptile
@ryangiomi
@serpentjoe
@tejacques

Reply to this email directly or view it on GitHub:
https://github.com/bemson/subetha-bridge/issues/1#issuecomment-152403634

Reply to this email directly or view it on GitHub:
https://github.com/bemson/subetha-bridge/issues/1#issuecomment-152406583

--

best,

bemson

@bemson , i'd say some solid buzzwords like "eventual consistency" and "append-only oplog" will be good here :) . At least that will make more clear, where to steal borrow dataflow principles (some DB design lectures).

Good to know @puzrin . These are definitely things I don't know of, and they capture the message well. If I ever get time, this will go into a wiki/blog somewhere. Anyone is free to repurpose this thread for the greater good!

@bemson before reading your posts i could not understand, that familiar patterns are ok for our case :) . Thanks for your work.
Bad news is that implementation requires a bit more that couple of code lines. Good news are that things become more clear & predictable:

There are implementations of such kind in other areas.
In IE10-11 append-style oplog could use IndexedDB instead of LS, to remove a lot of limitations and speed issues.
Using polling will add some freezes (polling interval), but will not reduce general throughput (well, with some specific LocalStorage limitations).

@bemson really interesting. I did something very similar to try and get a polyfill for IE8 StorageEvents -- IE8-EventListener. It's nice to see a lot of similar concepts going on which validate some design decisions I made there. I really hope you didn't have to spend as much time messing around figuring out the details there as I did, but I suspect you may have spent more.
That said, I re-read most of this thread, and noticed you had made some claims about IE9/10/11 with regards to StorageEvents firing and their contents. IE9-11's StorageEvent do have the standard key, oldValue and newValue on them, and running your jsbin reader and writer I was unable to get it to exhibit any dropped events on any of IE9/10/11, however because you're reading the value from localStorage (which hasn't necessarily updated yet by the time the event is received in IE) it shows the previous value. If you instead log the event's value, it gives the proper one data. This can be done with this snippet:
window.addEventListener('storage', function(e) {
  console.log(e);
});

And that works in IE9/10/11. I've only been able to get the issue of localStorage events fire twice or not at all in iframes to happen in IE11 -- I haven't replicated the behavior in IE9/10. The issue of events firing twice happens when localStorage is written to from an iframe, and the event not firing at all happens when an iframe points to Domain B with parent frame Domain A writes to localStorage.
The biggest obstacle I've encountered with using localStorage as a changelog in IE is that it can become expensive to look through every key in localStorage every time. If you have several tabs/iframes/windows open it quickly adds up. I've noticed that in IE, the relative order of keys returned by index from the key function does not appear to change if you add, remove, or change a value in localStorage. Because of this, I believe there is a lot of room for performance improvements by binary searching for a sentinel-value and reading new events from that marker.

@puzrin , my strategy has been to use LS for window registration and IDB for messaging. I do share one key for messaging, since all windows would then only have to inspect the latest IDB entries to catch up.
@tejacques , two points... ok, three.

I'm completely confused, about your claim about IEs storage event. Are you saying that it does have the key, newValue and oldValue properties??!
The jsbin reader test hits localStorage on purpose, because the storage event was empty in IE. If it's not, then I've just wasted 10 months of my coding life.
My particular project does require localStorage to be available in a (potentially) different-domain iframe. Most things work when we're talking same domain, but I'm a sucker for a challenge. :-p

@bemson

Yes, it does in IE9-11 -- check for yourself! These values were not present in IE8.
I'm aware of why it was doing it, but those properties are present, despite IE not even syncing localStorage at the time the event is received (hence reading from localStorage displays the previous value). It's not a waste however, because...
This is the major issue facing IE11 because it does not receive StorageEvents across parent domain boundaries (two tabs on different domains but with an iframe on a shared domain -- at least that's the way I understand it). I'm not sure if it is resolved in Edge, and as I mentioned, I'm not sure if it affects 9/10, but it does affect 11. Because of this, you have to treat all StorageEvents in IE11 (and possible 9/10 if it does affect those as well) as unreliable, and you do need to poll.

@tejacques - I'm completely befuddled by your findings, since they're counter to this 2009 article. You wouldn't be talking about your library's approach, where you retrieve the current value and capture it as the new storage item's value, right?
That said, I'll be sure to write a jsbin that simply dumps the storage event (from another window). I'll then place that in an iframe of another jsbin page, to see if the event details are still present. Unfortunately, I've got paid work to finish first! (I should have an update over the weekend.)

IE8 was released March 19 2009, IE9 wasn't released until March 14 2011. Since that article is from 2009 and mentions IE8, I've got to believe it's referring specifically to IE8. I do want to stress however that a method similar to what you've written is necessary with IE11 to be fully compliant when working with iframes.

@tejacques Sweet ba-jesus, thank you! You're right in that it doesn't remove the need for my findings, but it feels good to know that IE10 might be more standards compliant than I'd thought. Plus, I wouldn't have to fork my code as much as I have already. I will definitely research this further!

@tejacques , IE9 & IE10 show no poor storage event objects or performance. It's amazing how that little bit of misinformation led me to down such a dark path.
I wrote two test files on cssdeck.com. One writes to storage, and the other captures storage events. I even allowed for the event to be captured from the document or window - since that confused me as well.
Playing with these two pages, it's clear that IE9 and 10 do properly report the storage event. The capturing page inspects and ensures the event is usable - and it is!
I then, embedded these pages into jsbin.com, so as to mimic a third-party iframe scenario. The iframe writer and event listener both also work in IE9 & 10.
The cssdeck.com pages don't work in IE11; the user content is actually already in a different domain iframe (due to their setup). Therefore, I tested non-iframed jsbins versions of the same storage writer and event-listener. Thankfully, IE11 did fire the expected storage events when on the same domain.
Finally, to test polling IE 11 and iframes. I wrote a polling storage reader test (via cssdeck.com). It shows that IE11 can read localStorage via third-party iframes - but still doesn't receive  storage events.
Ok, so IE 11 (and perhaps Edge) are the outliers to this problem. That is better news than before, and thank you for catching my error. On the down side, the lack of events in IE11 still means plenty of code forking. Meanwhile, I'll update my IE10 implementation, to rely on the standards-based approach, going forward.
Whew! Are we there yet? :-p

@bemson see https://github.com/nodeca/tabex/labels/note for our records. Problems are with IE11 (cross-domain events) and iOS (timers + private mode). Also, probably you should know that iOS has problems with IndexedDB, but is ok with deprecated WebSQL.

@puzrin , thanks much. I'm aware of iOS and private mode, but my library (rather, my intended use of it) targets desktops and separate windows. If my project ever did target iOS, the IndexedDB note is good to know.

Ok. So, for my own sake, I checked the demo on IE11. The polling is definitely not great, but latency can be adjusted along with expectations. Anyway, here it is as an animated gif again.

The polling is definitely not great

You can initiate polling by both timer and "update" event (when available). Problem is not with polling, but with lack of effective oplog to replay events.

Last note in this, before my early weekend starts: IE Edge only fires storage events to iframes within the same domain and document. While no storage events are fired to other windows. Other windows seem to begin with a value, after load (and reload), neither re-reading localStorage or polling seem to work - iframe or not.

You should submit it as a bug to Microsoft. They are much more likely to fix it in Edge than IE11, and if it's the same code fix they may backport it.

@bemson just to be clear and so I am sure I understand correctly, what do you mean by "iframe or not" here
bemson:

neither re-reading localStorage or polling seem to work - iframe or not.

Does that mean if you have a tab with Domain A inside of which is an iframe to Domain B, that if you open a second tab to Domain B it will not receive Storage events or updates to localStorage from the iframe in the first tab?

@tejacques, in Edge, my module couldn't read the latest value of a storage item, whether read from a top-level or framed window.
The oddest thing is that even new windows would not see the latest value. This sequence of window storage actions, should illustrate what I've observed so far. (Windows A, B, and C windows have the same origin.)

Window
Key
Action
Result

A
foo
write 'bar'

B
foo
read
'bar'

A
foo
write 'zee'

A
foo
read
'zee'

B
foo
read
'bar'

B
random
write 'kick key'

B
foo
read
'bar'

C
foo
read
'bar'

Basically, even with newly created windows, if the key is set by another window, it would only ever receive the original value. It's as if changes to a storage are not flushed to disk - only captured when first created.

Unless someone beats me to it, I do intend to verify and file a bug for this behavior - over the weekend.

Ok wow, so it's just that localStorage is completely broken in Edge. Here are some issues open, but none give a great description:
https://connect.microsoft.com/IE/feedback/details/1694027/storage-event-not-firing-after-a-change-of-local-storate-windows-10-edge-browser
https://connect.microsoft.com/IE/feedbackdetail/view/1798743/localstorage-bug

Yes @tejacques, that's pretty bad and neigh impossible to fix - considering how the OS is a likely culprit. (Plus, it seems the IE 11/Edge quirks are throwbacks from IE8.)
Unfortunately, my Bridge module does not use IndexedDB for networking windows, nor the "peers" they create. Instead, I only use IDB for relaying messages. The registry of "bridges" and "peers" is wholly maintained in localStorage.
I did try to use IDB only, but the async logic and numerous failure-paths proved very difficult to code (at least, without Promises). As well, there were other goals for my API and protocol's design which felt best served by simple(r), local, synchronous logic.
Now, in hindsight, I may have to return to a network that uses storage events as a signal to read IDB. (I'd poll localStorage in IE11, and cookies in IE Edge.) I wanted to place my network map in one browser feature, rather than fork it for localStorage and IDB... A silver lining may be to depend on a persistence normalization library, like Storage.js)... It's all bad, so far. :-(

@bemson are there any requirements for msg delivery latency between local tabs? Need some info from the real world for investigations. My projects have no special requirements, and even 1000ms is ok.

My project has no latency requirements either @puzrin . My goal in addressing IE is message integrity - ensuring that what is sent by one client can be delivered to it's target(s) - also, of course, in the intended order.

I'm not exactly the most disciplined coder, so I have used localStorage (instead of IDB) to avoid the many negative async paths that are common to networking logic, like canceling a request to join a network.
I had originally planned to use IDB as a network registry, but it proved challenging without using a third-party library (which I loath to add due to footprint and performance concerns). As it stands now, I've reduced the number of IDB stores from 3 to 1, which has been a good thing so far...
For now, I'm trying to wrangle localStorage in IE10 & IE11. Despite @tejacques timely reminder, IE10 has some edge case behavior that shows things aren't working quite as expected still. I hope to update this thread more often with test links, etc, to demonstrate my findings.

I put a large amount of effort into testing this last night. Still need to go over some edge cases in IE10/11, but I will update with my findings later.
One important thing however is that the latency for Edge (which I found was synchronizing) was extremely poor. It took 3s for changes in one window to reflect in another, meaning a 6s latency.

I haven't had much time to put towards this, and it seems I'm back in "survey mode" for IE. But, I completed some test pages that assess how well localStorage reads and writes, in various contexts.

frameless read/write/observe
same-origin iframe w/r/o
different-origin iframe w/r/o

I'm not sure if the dabblet link expires - I signed up, hoping it would not.
It's tedious work, but I hope to provide a table of results for each test setup, per IE browser (10, 11, then Edge, across Windows 7 thru 10). Each page would be tested against itself, in another window. Then, the page would be tested alongside two windows for the other urls.
The test permutations are mind-numbing. But it feels like the only way to get a handle on this beast and provide the best depth of insight for developers and the IE browser team. I hope to update this thread with some results sooner, rather than later.

Ok, I didn't document my findings, but it looks like all IE browsers and platform combinations will see changes to localStorage in other windows - be it a same/different-origin iframe. Thats the good news... The bad/expected news is that storage events won't always work, based on the combination of browser and platform.
In short, without events, you'll have to do a few things to get localStorage to work:

poll localStorage for changes
kick localStorage to force it in sync
test that keys exist, whether you scan by length or a for-in loop

There is still worst news that I have yet to corroborate, regarding performance thresholds where none of these techniques seems to work any more. Another big hassle will be feature-testing the browser, to determine what approach to take. (I've been comfortable with UA checks, so far.)
I'll work to produce that table of results during this week, to corroborate my observations.

Below, two tables represent two browser windows, featuring versions of IE and the Windows OS. One window writes to localStorage, and the other listens for storage events. The tables lists, after updating localStorage, how the latest value was retrieved.
Reader window results

Browser
Platform
Frameless
Framed
same-origin
diff-origin

IE 9
Win 7
ok
ok
ok

IE 9
Win 7
ok
ok
ok

IE 9
Win 7
ok
ok
ok

IE 9
Win 7
ok
ok
ok

I finally have some data to share.
Data
I've compiled two tables, representing two browser windows, in various IE and Windows OS versions, for three scenarios: a frameless page, a same-origin iframe, and a different-origin iframe.
One table is for the window that is observing localStorage changes (the "observer"). The other is for the window manipulating localStorage (the "writer"). Here is an explanation of methods, used to observe/retrieve localStorage.

event - A storage event occurred. This, of course, is the ideal approach.
read - No storage event occurred. A storage item had to be retrieved manually.
kick - No storage event occurred. LocalStorage had to be synced manually (or "kicked") before a storage item could be retrieved manually.

Results for the Observing Window
Below lists how an observing window fared while tracking localStorage changes (by another window).

Browser
Platform
Frameless
Same-origin
Diff-origin

IE 9
Win 7
event
event
event

IE 10
Win 7
event
event
event

IE 10
Win 8
event
event
event

IE 11
Win 7
event
read
kick

IE 11
Win 8
event
read
kick

IE 11
Win 10
event
read
kick

IE Edge
Win 10
kick
event
kick

Results for the Writing Window
Below captures how the writing window fared in tracking it's own localStorage changes.

Browser
Platform
Frameless
Same-origin
Diff-origin

IE 9
Win 7
event
event
event

IE 10
Win 7
event
event
event

IE 10
Win 8
event
event
event

IE 11
Win 7
event
event
event

IE 11
Win 8
event
event
event

IE 11
Win 10
event
event
event

IE Edge
Win 10
read
event
kick

Takeaways
Surprisingly, under the right conditions, IE 11 and Edge will support storage events. Unfortunately, when depends on more than one factor (e.g., the browser version alone is not enough).
In my mind, this means two things:

Any cross-browser storage logic, must use a safe(r) data strategy (e.g., no shared keys, etc.)
Feature detection is the best way to ensure any storage-event shim uses native capabilities, when possible.

To date, I've been reticent to fork my own storage-shim using feature-detection, but the usefulness of native storage events is too compelling. As I feared - because I'm lazy - I will have to create a separate storage-events shim for localStorage.

Everyone is welcome to review my localStorage test pages. If I'm doing anything wrong or unadvisable, do let me know so I can capture and share more accurate information.

So, I tried my hand at feature detecting storage events. This page is supposed to conclude the same event/read/kick result as my manual research demonstrated.
In short, it seems that a lone window & script can not determine a browser's storage-event capabilities. The issue is that IE's localStorage behaves differently between iframes, than it does between windows. This insurmountable fact means a single script won't be enough.
That said... I will try one more test, which attempts to see if sibling iframes behave like sibling windows. My theory is that the storage event bubbles up (from child to parent frames), to the root window. Perhaps then, a single script can monitor two iframes, the way I monitored two windows.
If all that fails - which it likely will - I'm going to forgo a separate localStorage fix/module/repo. I will then add to the browser sniffing logic of my Bridge module, so it at least uses the best storage-event approach for a given IE + platform + framed/origin combination. It won't be an elegant solution, but it will hopefully solve this issue.

Oh well, having one iframe listen to another (in the same page) doesn't match my results either. It looks like a browser's storage event behavior can't be feature-detected via a single page. I'll double check my findings earlier, then use the best localStorage strategy based on IE version, origin, and platform... UGH!

More LocalStorage Setbacks
Some recently pushed updates, reveal even more troubling observations on IE's localStorage implementations: dropped changes and events.
Discovering the Death Knell of IE11 and IE12
My non-event key strategy centered around scanning localStorage, periodically, in order to observe "network" activity. This worked, but was incredibly costly. (It actually caused IE11 to crash consistently, with just two or three windows.) Instead of scanning blindly, I added a timestamp key, which - when updated - would cause all bridges to scan localStorage. This reduced the number of scans and held a lot of promise towards finalizing an IE solution.
I tested the approach using an ad hoc stress-test: quickly opening and closing about 30-50 windows. Effectively, I was trying to break my own networking logic. It did. . . break, that is.
While the browser no longer crashed, random windows would appear to be out of sync (showing a different number than the other bridges). Inspecting the broken bridge's localStorage data confirmed that that window's localStorage had different values than the others. Worst still, my "kick" technique no longer worked.
The out-of-sync windows (sometimes) responded to existing and new window activity. But, they maintained one or more ghost windows - bridges that were no longer visible to the rest of the network.
Dropped Changes
Stepping through the scan logic revealed that IE11 and IE12 would receive key removal events, for the bridge's own key. That is, after writing it's own key-identifier to localStorage, a scan could not find that key anymore. So, under specific conditions, IE11 writes to localStorage would not be present anymore: the change was dropped.
I'm pretty sure this is a simple - yet insurmountable - i/o issue. I've discovered that two bridges/windows, writing to localStorage can clobber the other's writes. Such, that reconciling the last write could fail, making it seem like it's key was removed.
The result is that the bridge is a ghost on the network. Other bridges may (or may not) see it, and it can see whichever bridges successfully added their key-identifiers. In that environment, some bridges can see each other and others can't.
Self-Destructing Ghost Bridges
A solution to ghosted bridges is to have it self-destruct. This is only for reducing harm to the network; only bridges that know they are on the network can use the network.
I’ve updated the original Bridge demonstration, so that the window turns red if the bridge is "ghosted". You’re welcome to stress test this page in IE 11 and 12, on Win7, 8, and 10. Even this kind of ad hoc test should give you sense of how reliable (rather unreliable) localStorage essentially is, under the worst conditions.
In the end, dropped changes in IE are insurmountable, whatever their frequency.
For my particular library, the only bright side is that the client could relaunch a ghosted bridge. But that would result in an intermittent client connection, which is never good for the dependent app code.
Dropped Events in IE 10
Though IE10 supports events, an apparent symptom of dropped changes are dropped events. (I don't doubt that IE10 drops changes the same as IE11+.) The result produces zombie bridges, where a bridge no longer receives updates about other bridges, since the event about that bridge never occurs. I could write code that reconciles with localStorage, but that leaves event-based logic open to the out-of-sync state of localStorage (something I've been trying to avoid).
I don't have a workaround for dropped events. In fact, if Chrome and other storage-event compliant browsers respond to high i/o demands like this, then storage events were probably never a viable, scalable option.
A silver lining: IDB, Blobs, & Workers
Polling IndexedDB may be my last hope. Surprisingly, you can access IDB in Workers on Chrome, Firefox, and IE 10-12. More importantly, this fiddle demonstrates accessing IDB via a Worker in a different-origin iframe!
So, while polling anything, and async everything is not ideal, it's better when done via Workers (in theory, at least) Additionally, if it all works in a dynamic, Blob-sourced Worker, it could be done without a separate script file. (Thanks @davidrekow!)

I thought to properly isolate the i/o issue, by removing my library code...
This is a rudimentary localStorage page, which writes a storage key then tests for it's presence 3 seconds later. If the key is not present, it makes the page background red.
I thought the test was too simple, and wouldn't fail - as if, more operations or localStorage manipulation were needed beforehand. However, after opening about 36 windows in IE Edge (holding down "ctrl-n"), several windows had red backgrounds.

In IE11 WIn8.1, one window was red, after opening about 25 windows.

To some, these ad hoc results could be viewed as an edge-case. It's evidence of how localStorage - let alone localStorage events - is not the ideal backbone for my cross-platform networking architecture. Any thoughts?

Any thoughts?

I think, attempts to make those events work is a loss of time. You still can implement eventually consistent oplog, without events.

That's depressing. I'm curious though if StorageEvents are sent properly/received properly. My best guess at this point is "no" and that IE/Edge are just broken with regards to localStorage.
I really, really hope Microsoft fixes them. It's at least worth opening new issues and linking to your findings.

@puzrin, you certainly could use IDB for oplog purposes. LocalStorage is going to fail you (at least, in IE). I'm not sure if I'll shoot for an operations-log or change-monitoring approach, in IDB. Likely depends on the goals of your library.
@tejacques , I've registered with msdn and will get on that ticket you've mentioned. Regarding events, I noted how some test pages would fall out of sync in IE 10.
I'm certain events are dropped, and for the same i/o reason: dropped changes (can not be broadcast). Here, I got the same result in IE 10+Win8, as I did IE 11+Win8.

Of course, Windows 8/10 may be the tie that binds. I have yet to produce similar results in any IE on Win7.
Oddly enough, I started the test at 500ms. Turned out that the longer the delay made it easier to identify dropped storage writes.

Finally filed an issue with Microsoft, regarding dropped localStorage writes (#2141793). I'm sure there is something else here that I'll file yet another ticket about...

Merry 2016 all. I'm finally back on the horse with this project. After discovering how localStorage is a poor candidate for window to window networking (for IE, at least), I'm pursuing the append-only log idea that @puzrin floated awhile back, via #5 .
Though the focus of this issue was wavered, I'll keep it open until IE networking is solved. Thanks for everyone's contribution and support with getting localStorage to behave in IE. You're strongly encouraged to provide guidance with the new IndexedDB approach!


refactor: prepare spending fidelity bond

This is a refactoring in preperation of #547
Why? Because the follow-up PR would contain too much changes to be properly reviewed.
The change set may seem a little bit overwhelming, but this PR is in fact just renaming and moving stuff around.
e.g.

more generic components  (especially regarding passing translations strings) that will be used in PR for #547
add missing i18n translation keys
add propertyutxosByJar to WalletInfo (because it is used quite frequently and should only be created once)
renaming account -> jar wherever possible

Also, it adds a globals.d.ts file to contain all app-wide used TypeScript types (currently only holding JarIndex).
One small bugfix is included: Prevent an error while creating a Fidelity Bond from displaying a loading indicator indefinitely.
Before this fix, when creating a FB failed, a user could not restart the procedure. After this PR is applied, a user can properly restart the flow in order to try creating a FB again.
How to test?
Try creating a fidelity bond with a coinbase transaction that has less than 100 confirmations (execute ./docker/regtest/mine-block 1 ${address_from_jar_A}) without this fix applied (e.g. from the current master branch) and notice that the resulting error prevents users from restarting the "create FB"-flow. Then apply this fix (by checking out this branch git checkout prepare-spend-fb) and try again. This time the error should be displayed and a button to "Start over" is presented.

utACK


DBeaver shows wrong dates for old dates

System information:

Operating system (distribution) and version - Mac 10.15.5
DBeaver version - Community Editition 7.1.3
Additional extensions - None

Connection specification:

Database name and version - Ocient
Driver name - Ocient JDBC
Do you use tunnels or proxies (SSH, SOCKS, etc)? No

Describe the problem you're observing:
We're seeing incorrect dates for old dates like the year 1000.
The problem is that java.util.Date uses historical dates, and switches from the Gregorian calendar to the Julian calendar on October 15, 1582.  However, the SQL standard says that dates are supposed to use the proleptic Gregorian calendar (which means to pretend the Gregorian calendar has always been in use).
JDBC drivers should implement a subclass of java.sql.Date to handle this correctly.  And its toString() method will print the correct dates.
But, it appears that DBeaver uses SimpleDateFormat to format dates instead of toString().
The problem is that SimpleDateFormat just ends up calling getTime() on the date, and then pulls a generic GregorianCalendar to do the formatting, and all of a sudden it's all wrong again.
To fix this, DBeaver code might set the calendar on SimpleDateFormat's before doing formatting to enable the proleptic Gregorian calendar, like this:
GregorianCalendar cal = new GregorianCalendar();
cal.setGregorianChange(new java.util.Date(Long.MIN_VALUE));
sdf.setCalendar(cal);#### Steps to reproduce, if exist:
Include any warning/errors/backtraces from the logs

If possible we would greatly prefer that if it's an Ocient DB, that instead of using SimpleDateFormat at all, DBeaver just use toString() on the underlying type Time = XGTime, Timestamp = XGTimestamp, Date = XGDate. Because we are adding nanosecond precision to Time and Timestamp and Dbeaver will still not display that unless it uses toString().
We, Ocient and DBeaver, can add this to our other discussions outside of this chat as well.

Thank you for the report. I do not have Ocient database and I've tried MySQL and PostgreSQL:

MySQL: date type is ok, timestamp gives an error on any date under 1970 year (e.g. Kseniia.contacts):
SQL Error [1292] [22001]: Data truncation: Incorrect datetime value: '1960-06-30 00:00:00' for column 'created_date' at row 1
Data truncation: Incorrect datetime value: '1960-06-30 00:00:00' for column 'created_date' at row 1
PostgreSQL: date and timestamp types are ok.

Thanks for the help! It needs to be on a very old date like 1400. It'll work for 1960.

Unfortunately we can't reproduce or fix this because it is not applicable to MySQL or PG.
Seems to be an Ocient-specific issue.


Add ability to serialize symbol keys in JS_WriteObject

Fixes: https://github.com/quickjs-ng/quickjs/issues/481

@bnoordhuis Is this what you had in mind? If so, I'll add the missing part to serialize symbols directly, as in bjson.write(Symbol('foo')) and push it over the finish line.

Symbol.toStringTag is a well-known symbol but would it also work for e.g. Symbol('#frob')?

Symbol.toStringTag is a well-known symbol but would it also work for e.g. Symbol('#frob')?

I believe so, I did consider it:
qjs > new Uint8Array(bjson.write({[Symbol('#frob')]: "42"}))
Uint8Array(16) [ 15, 1, 3, 10, 35, 102, 114, 111, 98, 8, 1, 2, 7, 4, 52, 50 ]

The "problem" is that normal symbols won't match after being deserialized, but I'd say that is expected, right?

Yes, I'd say so; Symbol("#frob) != Symbol("#frob)
Symbol.for("#frob) == Symbol.for("#frob) on the other hand so maybe it should work for those. You probably have to work in a bit in the wire format to record whether it's JS_ATOM_TYPE_GLOBAL_SYMBOL vs. JS_ATOM_TYPE_SYMBOL.
You're more or less handcrafting the symbol from the wire data now, right? Maybe it's easier to use JS_NewSymbolInternal()?

I'm passing in the atom and the type so the distinction is already there. I'm only special casing the internal atoms.
I'll continue tonight!

@bnoordhuis Updated, PTAL!


[w5.3b][W13-A1]Dominic Kenn Lim

Extract commonalities from Phone, Email, Address to Contact class

@dominickenn
This is quite a messy PR.
Avoid accumulating unrelated changes into a single PR.
Tip: If you are currently on branch1 and want to create a new branch2, you need to switch back to master branch before creating branch2. If not, changes in branch1 will appear in the PR created from branch2 later.
Some comments added. Please ack & close the PR after reading comments.

Acknowledged.


Option to disable UUID generator

Hi, it would be nice to have option to disable UUID generator. Thing is that ArangoDB uses number keys that are sufficient for most apps and UUIDs are not necessary and definitely less URI friendly.
Or is there reason why is this solution used?

Honestly I put it in because our security operation team preferred that we used standardized ID generation across all our databases, and in general they didn't like the sequential IDs.
I'd be more than happy to remove it. Instead, people can drop in their unique IDs through the before creation hook.
If you want to make the change and submit a PR I'd be happy to merge it in. Otherwise I won't be able to get to it until tomorrow morning. I'll happily make the change then though if you haven't done it


deep learning applications readme patch breaks tensorflow>=1.12

Hi,
I'm trying to run geomstats deep learning applications on Ubuntu 18.04 LTS using my RTX 2080 GPU. I systematically encounter an error that suggests the patch on variables.py described in the repo's readme breaks my tensorflow. Hardware and drivers description:

Ubuntu LTS 18.04
NVIDIA driver version 418.56 (proprietary)
CUDA 10.0
cuDNN 7.5.0 (installed from .deb files)
RTX 2080 GPU
Ryzen 2700x CPU

I have one possible explanation, although I find it absurd, but in case it saves us a lot of time: do the deep learning applications only work with tensorflow version 1.8 ? Because the requirements.txt of geomstats says tensorflow>=1.12, whereas the applications repo readme only mentions 1.8.
I tried the following installation steps, with:

tensorflow-gpu 1.13.1
tf-nightly-gpu 1.14.0.dev2019032201
tensorflow 1.13.1 (basic CPU-only version, just in case it was GPU-related in some way)

create a Python 3 (3.6.7) virtual env
install one of the three previously mentioned tensorflow and geomstats (1.15), with pip
run tensorflow toy code to check if unpatched virtual env tensorflow works (it does this far in the steps, for all three versions of tf considered)
patch virtual env tensorflow according to the applications repo's instructions
install patched keras thanks to the previous step instructions, doing so with a modification of the original instructions since the target is the virtual env and not the system, hence use python setup.py install and not the readme sudo version. The shell return allows to check which Keras was installed and updated, and more specifically the exact version and whether this was indeed done in the virtual env

Using /home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages
Finished processing dependencies for Keras==2.1.6

try ./run.sh or python <MNIST or CIFAR example>.py and encounter same error (see at the end of this issue for complete error message)
try tensorflow toy code that worked in the virtual env before patching, and encounter same error
conclude that the patch broke tensorflow according to related tf repo issue
 (they advise re-installing tensorflow to solve this error, but this basically means cancelling the patch here...)

One should note that the error specifically mentions the unique file that was modified by the patch virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/ops/variables.py.
In addition, I tried both installing Keras with pip right after tensorflow when launching the venv, and only installing Keras with the provided setup.py script. All ended with the same error.
Complete error message:
geomstats_examples/applications/deep_learning$ python cifar_10_resnet_hypersphere.py
Using TensorFlow backend.
Traceback (most recent call last):
  File "cifar_10_resnet_hypersphere.py", line 13, in <module>
    import keras
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/Keras-2.1.6-py3.6.egg/keras/__init__.py", line 3, in <module>
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/Keras-2.1.6-py3.6.egg/keras/utils/__init__.py", line 6, in <module>
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/Keras-2.1.6-py3.6.egg/keras/utils/conv_utils.py", line 9, in <module>
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/Keras-2.1.6-py3.6.egg/keras/backend/__init__.py", line 84, in <module>
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/Keras-2.1.6-py3.6.egg/keras/backend/tensorflow_backend.py", line 5, in <module>
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 63, in <module>
    from tensorflow.python.framework.framework_lib import *  # pylint: disable=redefined-builtin
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/framework/framework_lib.py", line 52, in <module>
    from tensorflow.python.framework.importer import import_graph_def
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 28, in <module>
    from tensorflow.python.framework import function
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 37, in <module>
    from tensorflow.python.ops import resource_variable_ops
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 41, in <module>
    from tensorflow.python.ops import variables
  File "/home/mbauw/Documents/virtual_env/geomstats_env_third/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 32, in <module>
    from tensorflow.python.training import checkpointable
  ImportError: cannot import name 'checkpointable'

Hi, and sorry for the delay in the response,
That's right, you will need a specific version of tensorflow and a manual patch. We know this is very suboptimal and are working on a patch right now.
Can I ask what precisely you're trying to check and which usecase? Are you trying to do optimization on a specific manifold?
Best,
Johan

Can I ask what precisely you're trying to check and which usecase? Are you trying to do optimization on a specific manifold?

If those questions are not directly related to the specific issue of this post as I understand them: I plan on using SPD matrices to do classification on signal and image processing to begin with. I'm not sure at the moment whether I will, in addition to transforming my data into SPD matrices, constrain parameters to specific manifolds, but if I manage to get geomstats to do that for me I'm interested in trying it and playing with the possibilities to see how it impacts the networks performances. Since I'm discovering Geometric DL, I'm also interested at looking at how you actually implemented the metrics etc.
To conclude this specific issue, unless I'm mistaken, your answer is simply that at the moment, constraining the parameters of a keras neural network to a chosen manifold is a geomstats feature only available using tensorflow 1.8 then ? If yes, please excuse the issue opening, since it's kinda explained in the application readme.

I think that in your case you don't need to use this example as a base - you should probably be looking at the SPD application example.
My understanding is that your manifold is on the activations, not the weights of the network.
You are right that this example is currently only for tensorflow 1.8. Indeed if we wanted to have this in all the tensorflow we would need to send the patch upstream or actively maintain it which would probably be a full time job. We are open to contributions or ideas if you think that there are better ways to keep this working in all versions.


Updating documentation from make install to make install-crds

What happened:
The documentation needs to be updated as our Makefile has changed.
https://github.com/kudobuilder/kudo/blame/master/README.md#L42 & https://github.com/kudobuilder/kudo/blame/master/docs/getting-started.md#L25 referencing wrong Make phony with make install.
What you expected to happen:
It should be make install-crds.
How to reproduce it (as minimally and precisely as possible):
$ make install
make: *** No rule to make target `install'.  Stop.

Anything else we need to know?:
Using make install-crds works just fine.
$ make install-crds
kubectl apply -f config/crds
customresourcedefinition.apiextensions.k8s.io/frameworks.kudo.k8s.io created
customresourcedefinition.apiextensions.k8s.io/frameworkversions.kudo.k8s.io created
customresourcedefinition.apiextensions.k8s.io/instances.kudo.k8s.io created
customresourcedefinition.apiextensions.k8s.io/planexecutions.kudo.k8s.io created

Environment:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-26T00:04:52Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}

Kudo version (use kubectl kudoctl version):

KUDO Version: version.Info{GitVersion:"v0.1.0", GitCommit:"c0f73df1226f4296dfdbf4c77ac33a7b27c4c846", BuildDate:"2019-04-22T15:33:04Z", GoVersion:"go1.12.4", Compiler:"gc", Platform:"darwin/amd64"}

Framework:
Frameworkversion:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

As the only action that make install-crds takes is kubectl apply -f config/crds I think it would be better to document running kubectl instead - the use of a make target implies complexity to the user that isn't actually there.

I went over the repo and it seems to be fixed in all places so closing