Integration stopped working for Deye. Querying [514 - 558] failed with exception

Hi,
After couple months of work the integration suddenly stooped working today.
Looking at the logs the communication is working but it seem that the data cannot be decoded?
All entities went into Unknown state
status_connection showing as disconnected
Inverter serial: 23**
Inverter model: 12K-SG04LP3-EU
Logger firmware: LSW3_32U5406_1.06
2024-05-08 21:10:05.862 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Starting to query for [6] ranges...
2024-05-08 21:10:05.862 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Querying [3 - 89]...
2024-05-08 21:10:05.862 INFO (SyncWorker_1) [custom_components.solarman.solarman] Connecting to solarman data logger <IP_ADDRESS>:8899
2024-05-08 21:10:05.866 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Socket setup completed... <socket.socket fd=20, family=2, type=1, proto=6, laddr=('<IP_ADDRESS>', 39950), raddr=('<IP_ADDRESS>', 8899)>
2024-05-08 21:10:05.867 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] SENT: a5 17 00 10 45 80 00 03 eb 4c a6 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 03 00 03 00 57 f4 34 54 15
2024-05-08 21:10:06.203 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] RECD: a5 c1 00 10 15 80 7d 03 eb 4c a6 02 01 2d 99 d6 00 08 1e 00 00 d0 26 65 65 01 03 ae 32 33 30 39 31 31 30 35 30 32 00 00 00 00 00 00 18 07 00 00 00 00 20 06 11 40 00 00 10 01 c0 3e 00 00 d4 c0 00 01 02 03 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff ff 18 05 08 15 08 14 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 00 01 00 64 00 64 ff ff 00 00 04 4c 00 01 00 00 ff ff ff ff ff ff 00 00 00 01 ff ff ff ff ff ff 4b cc 9a 15
2024-05-08 21:10:06.204 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Querying [3 - 89] succeeded
2024-05-08 21:10:06.205 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Querying [514 - 558]...
2024-05-08 21:10:06.205 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] SENT: a5 17 00 10 45 81 00 03 eb 4c a6 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 03 02 02 00 2d 25 af d8 15
2024-05-08 21:10:06.487 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] RECD: a5 6d 00 10 15 81 7e 03 eb 4c a6 02 01 2d 99 d6 00 09 1e 00 00 d0 26 65 65 01 03 5a 00 49 00 2c 31 67 00 00 2d 8a 00 00 00 02 02 f4 66 e1 00 00 5f 88 00 00 00 d0 8d 43 00 00 04 10 00 00 00 00 00 00 00 00 9c 47 00 00 00 00 00 00 00 00 00 00 04 e2 05 99 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 00 01 00 05 00 00 00 00 00 00 00 00 00 00 00 00 83 74 61 15
2024-05-08 21:10:06.488 WARNING (SyncWorker_1) [custom_components.solarman.solarman] Querying [514 - 558] failed with exception [ValueError: Invalidate complete dataset (Daily Production ~ 104.0)]
2024-05-08 21:10:06.489 INFO (SyncWorker_1) [custom_components.solarman.solarman] Disconnecting from solarman data logger <IP_ADDRESS>:8899
2024-05-08 21:10:06.491 WARNING (SyncWorker_1) [custom_components.solarman.solarman] Querying [514 - 558] failed, [1] retry attempts left
2024-05-08 21:10:06.491 INFO (SyncWorker_1) [custom_components.solarman.solarman] Connecting to solarman data logger <IP_ADDRESS>:8899
2024-05-08 21:10:06.495 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] Socket setup completed... <socket.socket fd=20, family=2, type=1, proto=6, laddr=('<IP_ADDRESS>', 39962), raddr=('<IP_ADDRESS>', 8899)>
2024-05-08 21:10:06.495 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] SENT: a5 17 00 10 45 4d 00 03 eb 4c a6 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 03 02 02 00 2d 25 af a4 15
2024-05-08 21:10:06.668 DEBUG (SyncWorker_1) [custom_components.solarman.solarman] RECD: a5 6d 00 10 15 4d 7f 03 eb 4c a6 02 01 2d 99 d6 00 09 1e 00 00 d0 26 65 65 01 03 5a 00 49 00 2c 31 67 00 00 2d 8a 00 00 00 02 02 f4 66 e1 00 00 5f 88 00 00 00 d0 8d 43 00 00 04 10 00 00 00 00 00 00 00 00 9c 47 00 00 00 00 00 00 00 00 00 00 04 e2 05 99 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 00 01 00 05 00 00 00 00 00 00 00 00 00 00 00 00 83 74 2e 15
2024-05-08 21:10:06.669 WARNING (SyncWorker_1) [custom_components.solarman.solarman] Querying [514 - 558] failed with exception [ValueError: Invalidate complete dataset (Daily Production ~ 104.0)]
2024-05-08 21:10:06.669 INFO (SyncWorker_1) [custom_components.solarman.solarman] Disconnecting from solarman data logger <IP_ADDRESS>:8899
2024-05-08 21:10:06.671 WARNING (SyncWorker_1) [custom_components.solarman.solarman] Querying [514 - 558] failed, [0] retry attempts left
2024-05-08 21:10:06.671 WARNING (SyncWorker_1) [custom_components.solarman.solarman] Querying registers [514 - 558] failed, aborting.
2024-05-08 21:10:06.671 DEBUG (SyncWorker_1) [custom_components.solarman.sensor] No value recorded for PV1 Power
2024-05-08 21:10:06.674 DEBUG (SyncWorker_62) [custom_components.solarman.sensor] No value recorded for PV2 Power
2024-05-08 21:10:06.675 DEBUG (SyncWorker_22) [custom_components.solarman.sensor] No value recorded for PV1 Voltage
2024-05-08 21:10:06.677 DEBUG (SyncWorker_28) [custom_components.solarman.sensor] No value recorded for PV2 Voltage
2024-05-08 21:10:06.678 DEBUG (SyncWorker_35) [custom_components.solarman.sensor] No value recorded for PV1 Current
2024-05-08 21:10:06.679 DEBUG (SyncWorker_35) [custom_components.solarman.sensor] No value recorded for PV2 Current
2024-05-08 21:10:06.681 DEBUG (SyncWorker_27) [custom_components.solarman.sensor] No value recorded for Daily Production
2024-05-08 21:10:06.683 DEBUG (SyncWorker_15) [custom_components.solarman.sensor] No value recorded for Total Production
2024-05-08 21:10:06.684 DEBUG (SyncWorker_7) [custom_components.solarman.sensor] No value recorded for Daily Battery Charge
2024-05-08 21:10:06.685 DEBUG (SyncWorker_2) [custom_components.solarman.sensor] No value recorded for Daily Battery Discharge
2024-05-08 21:10:06.687 DEBUG (SyncWorker_37) [custom_components.solarman.sensor] No value recorded for Total Battery Charge
2024-05-08 21:10:06.689 DEBUG (SyncWorker_8) [custom_components.solarman.sensor] No value recorded for Total Battery Discharge
2024-05-08 21:10:06.690 DEBUG (SyncWorker_40) [custom_components.solarman.sensor] No value recorded for Battery Power
2024-05-08 21:10:06.692 DEBUG (SyncWorker_42) [custom_components.solarman.sensor] No value recorded for Battery Voltage
2024-05-08 21:10:06.693 DEBUG (SyncWorker_43) [custom_components.solarman.sensor] No value recorded for Battery SOC
2024-05-08 21:10:06.694 DEBUG (SyncWorker_10) [custom_components.solarman.sensor] No value recorded for Battery Current
2024-05-08 21:10:06.696 DEBUG (SyncWorker_41) [custom_components.solarman.sensor] No value recorded for Battery Temperature
2024-05-08 21:10:06.697 DEBUG (SyncWorker_17) [custom_components.solarman.sensor] No value recorded for Total Grid Power
2024-05-08 21:10:06.699 DEBUG (SyncWorker_51) [custom_components.solarman.sensor] No value recorded for Grid Voltage L1
2024-05-08 21:10:06.700 DEBUG (SyncWorker_48) [custom_components.solarman.sensor] No value recorded for Grid Voltage L2
2024-05-08 21:10:06.702 DEBUG (SyncWorker_52) [custom_components.solarman.sensor] No value recorded for Grid Voltage L3
2024-05-08 21:10:06.703 DEBUG (SyncWorker_50) [custom_components.solarman.sensor] No value recorded for Internal CT L1 Power
2024-05-08 21:10:06.705 DEBUG (SyncWorker_50) [custom_components.solarman.sensor] No value recorded for Internal CT L2 Power
2024-05-08 21:10:06.706 DEBUG (SyncWorker_9) [custom_components.solarman.sensor] No value recorded for Internal CT L3 Power
2024-05-08 21:10:06.708 DEBUG (SyncWorker_9) [custom_components.solarman.sensor] No value recorded for External CT L1 Power
2024-05-08 21:10:06.709 DEBUG (SyncWorker_21) [custom_components.solarman.sensor] No value recorded for External CT L2 Power
2024-05-08 21:10:06.711 DEBUG (SyncWorker_5) [custom_components.solarman.sensor] No value recorded for External CT L3 Power
2024-05-08 21:10:06.712 DEBUG (SyncWorker_56) [custom_components.solarman.sensor] No value recorded for Daily Energy Bought
2024-05-08 21:10:06.714 DEBUG (SyncWorker_57) [custom_components.solarman.sensor] No value recorded for Total Energy Bought
2024-05-08 21:10:06.715 DEBUG (SyncWorker_57) [custom_components.solarman.sensor] No value recorded for Daily Energy Sold
2024-05-08 21:10:06.717 DEBUG (SyncWorker_55) [custom_components.solarman.sensor] No value recorded for Total Energy Sold
2024-05-08 21:10:06.718 DEBUG (SyncWorker_55) [custom_components.solarman.sensor] No value recorded for Total Grid Production
2024-05-08 21:10:06.719 DEBUG (SyncWorker_59) [custom_components.solarman.sensor] No value recorded for Total Load Power
2024-05-08 21:10:06.721 DEBUG (SyncWorker_30) [custom_components.solarman.sensor] No value recorded for Load L1 Power
2024-05-08 21:10:06.722 DEBUG (SyncWorker_12) [custom_components.solarman.sensor] No value recorded for Load L2 Power
2024-05-08 21:10:06.724 DEBUG (SyncWorker_61) [custom_components.solarman.sensor] No value recorded for Load L3 Power
2024-05-08 21:10:06.725 DEBUG (SyncWorker_63) [custom_components.solarman.sensor] No value recorded for Load Voltage L1
2024-05-08 21:10:06.727 DEBUG (SyncWorker_25) [custom_components.solarman.sensor] No value recorded for Load Voltage L2
2024-05-08 21:10:06.728 DEBUG (SyncWorker_62) [custom_components.solarman.sensor] No value recorded for Load Voltage L3
2024-05-08 21:10:06.730 DEBUG (SyncWorker_23) [custom_components.solarman.sensor] No value recorded for Daily Load Consumption
2024-05-08 21:10:06.731 DEBUG (SyncWorker_23) [custom_components.solarman.sensor] No value recorded for Total Load Consumption
2024-05-08 21:10:06.732 DEBUG (SyncWorker_35) [custom_components.solarman.sensor] No value recorded for Current L1
2024-05-08 21:10:06.734 DEBUG (SyncWorker_27) [custom_components.solarman.sensor] No value recorded for Current L2
2024-05-08 21:10:06.736 DEBUG (SyncWorker_15) [custom_components.solarman.sensor] No value recorded for Current L3
2024-05-08 21:10:06.737 DEBUG (SyncWorker_7) [custom_components.solarman.sensor] No value recorded for Inverter L1 Power
2024-05-08 21:10:06.739 DEBUG (SyncWorker_2) [custom_components.solarman.sensor] No value recorded for Inverter L2 Power
2024-05-08 21:10:06.740 DEBUG (SyncWorker_37) [custom_components.solarman.sensor] No value recorded for Inverter L3 Power
2024-05-08 21:10:06.742 DEBUG (SyncWorker_8) [custom_components.solarman.sensor] No value recorded for DC Temperature
2024-05-08 21:10:06.743 DEBUG (SyncWorker_8) [custom_components.solarman.sensor] No value recorded for AC Temperature
2024-05-08 21:10:06.745 DEBUG (SyncWorker_8) [custom_components.solarman.sensor] No value recorded for Inverter ID
2024-05-08 21:10:06.746 DEBUG (SyncWorker_16) [custom_components.solarman.sensor] No value recorded for Communication Board Version No.
2024-05-08 21:10:06.748 DEBUG (SyncWorker_0) [custom_components.solarman.sensor] No value recorded for Control Board Version No.
2024-05-08 21:10:06.749 DEBUG (SyncWorker_0) [custom_components.solarman.sensor] No value recorded for Alert

Found the issue. Apparently the max Daily production cannot be greater than 100kWh. In my case it was 104kWh
 Querying [514 - 558] failed with exception [ValueError: Invalidate complete dataset (Daily Production ~ 104.0)]
Had to change the limit in deye_sg04lp3.yaml

Found the issue. Apparently the max Daily production cannot be greater than 100kWh. In my case it was 104kWh  Querying [514 - 558] failed with exception [ValueError: Invalidate complete dataset (Daily Production ~ 104.0)] Had to change the limit in deye_sg04lp3.yaml

Hey there,
Im faced with the same issue now on my Solarman integration, not sure why its happening now .
2 Questions - Where can i find the logs (running HA on Virtual box) for Solarman to try and isolate the problem.
2nd - What did you change your limit to?

Found the issue. Apparently the max Daily production cannot be greater than 100kWh. In my case it was 104kWh  Querying [514 - 558] failed with exception [ValueError: Invalidate complete dataset (Daily Production ~ 104.0)] Had to change the limit in deye_sg04lp3.yaml

Hey there,
Im faced with the same issue now on my Solarman integration, not sure why its happening now .
2 Questions - Where can i find the logs (running HA on Virtual box) for Solarman to try and isolate the problem.
2nd - What did you change your limit to?

There are couple ways to get logs:
First option:
Enabled debug logging on Solarman Integration

Let it run and then disable, once it is disabled you get logs downloaded

Second option:
Enabled debug logging on Solarman Integration
Then tail HA logs tail -f /config/home-assistant.log your path will be different depending where/how your HA.
This option lets you see logs as they happening.

Since different use different registers this is why definition files are used. In my case it is deye_sg04lp3.yaml. The files are located in /config/custom_components/solarman/inverter_definitions again the path depends where your HA is installed.
In the definition file there are some limits defined and if the values are out of bounds then integration assume that the values are no valid and thus shows then as Unknown . In my case it was

    - name: "Daily Production"
      class: "energy"
      state_class: "total_increasing"
      uom: "kWh"
      scale: 0.1
      rule: 1
      registers: [0x0211]
      icon: 'mdi:solar-power'
      validation:
        max: 100
        invalidate_all:

I've changed the max value to 150 saved changes and restarted HA. After that all started working.

Aah found it, thank you. This is the error im getting:

2024-05-09 13:20:37.844 DEBUG (MainThread) [custom_components.solarman] __init__.py:async_unload_entry({'entry_id': '24d74e25db56ffc51f735478d50b3ae4', 'version': 1, 'minor_version': 1, 'domain': 'solarman', 'title': '<IP_ADDRESS>', 'data': {'name': 'Solarman', 'inverter_host': '<IP_ADDRESS>', 'inverter_serial':<PHONE_NUMBER>, 'inverter_port': 8899, 'inverter_mb_slaveid': 1, 'lookup_file': 'deye_hybrid.yaml'}, 'options': {'name': 'Solarman', 'inverter_host': '<IP_ADDRESS>', 'inverter_serial':<PHONE_NUMBER>, 'inverter_port': 8899, 'inverter_mb_slaveid': 1, 'lookup_file': 'deye_hybrid.yaml'}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'unique_id': None, 'disabled_by': None})
2024-05-09 13:20:37.875 WARNING (MainThread) [homeassistant.components.energy.sensor] Found unexpected state_class None for sensor.solarman_daily_load_consumption
2024-05-09 13:20:37.905 ERROR (MainThread) [homeassistant.config_entries] Error unloading entry 192.168.10.** for solarman
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 772, in async_unload
result = await component.async_unload_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/solarman/__init__.py", line 28, in async_unload_entry
hass.data[DOMAIN].pop(entry.entry_id)
~~~~~~~~~^^^^^^^^
KeyError: 'solarman'

If it fails just on solarman_daily_load_consumption try to comment it out and see if it can pull other ones.


Write Python Output

Solves #36.

Once this is working, it will probably break most tests.

Failed concept.


Try to reduce literature

Description
The paper has 2 pages of literature for 1 page or text content.  Though most of the citations are in principle fine and relevant, some could possibly be removed.
Candidate from my perspective:

remove sentence "It enables researchers and environmental managers to gain deeper insights into the dynamic processes occurring in aquatic environments (Liu et al., 2021, 2023)." and refs to Liu

but there may be more...
Linked to https://github.com/openjournals/joss-reviews/issues/7025

As in the main review thread.  D. Katz said that JOSS does not limit refs.  So you can disregard this comment.  (But I still think the one sentence quoted above is not relevant)


add nil pointer check to ObjectStore collector

Add nil pointer check on ObjectStore collector to prevent crash when SecuritySpec is empty in CephObjectStore.
Signed-off-by: Umanga Chapagain<EMAIL_ADDRESS>

/cherry-pick release-4.12


Update the security intents in the context of container escape to host

Add examples of container escape intent to the examples folder


Setting default values for multiple dice

How can we specify values for individual dice by default?

I'm thinking the way to do this would be to introduce a prop called something like nextRollValues and pass in an array that matches the sequence of dice you are using.
The tricky part would be you'd need to clear that prop before the next roll.
@caseyjkey does this seem like something that would work for your case?


[Plugins] PluginRoutes seem not to be registered correctly in fresh.gen.ts

I want to update my fresh_openprops integration to use a real Fresh Plugin for the integration:
https://github.com/codemonument/deno_fresh_openprops/blob/main/src/fresh_openprops_plugin.ts
It has a factory function, which returns the plugin object like so:

export async function FreshOpenProps(rawOptions?: PluginOptions) {
  // Throws when option parsing fails
  const { cssInputPath, postcssModuleDirs, isProd, doPrefillCssCache } =
    PluginOptions.parse(
      rawOptions,
    );

  // Should only happen once, since this plugin is only initialized once
  await initPostcssInstance(postcssModuleDirs);

  if (doPrefillCssCache) {
    await prefillCssCache({ cssInputPath });
  }

  const postcssRoute = {
    path: "postcss/[...path]",
    handler: async (
      req: Request,
      ctx: HandlerContext,
    ): Promise<Response> => {
       // The handler code
    }
  } satisfies PluginRoute;

  return {
    name: "openprops",
    routes: [postcssRoute],
  } satisfies Plugin;

It integrates like this:
See my example in the repo for full code: https://github.com/codemonument/deno_fresh_openprops/blob/main/example/main.ts

// inside main.ts file

const openpropsPlugin = await FreshOpenProps({
  isProd: false,
  doPrefillCssCache: true,
  cssInputPath: "example/css",
  postcssModuleDirs: ["example/css_deps"],
});

await start(manifest, {
  plugins: [
    openpropsPlugin,
  ],
});

First, I had the openpropsPlugin object inlined but extracted it afterwards bc. I suspected the await of the factory function to mess up the plugin initialization.
Error Observations

The route /postcss/global.css cannot be found (404)
After Inspection: The route from the plugin is not even registered in fresh.gen.ts

// Content of fresh.gen.ts
import * as $0 from "./routes/index.tsx";

const manifest = {
  routes: {
    "./routes/index.tsx": $0,
  },
  islands: {},
  baseUrl: import.meta.url,
};

export default manifest;

Source: https://github.com/codemonument/deno_fresh_openprops/blob/main/example/fresh.gen.ts
What may be wrong here?

Hi @bjesuiter, thanks for logging this. I developed this (plugins supporting routes), so I guess I'm on the hook for debugging!  My test project (for when I'm developing the plugin) has a fresh.gen.ts which looks like this:
const manifest = {
  routes: {},
  islands: {},
  baseUrl: import.meta.url,
};

export default manifest;

even more blank than yours, and it works just fine. Even better, my plugin needs to be awaited, and I have my main.ts like this:
start(manifest, {
  signal: ac.signal,
  plugins: [await kvPlugin(options), twindPlugin(twindConfig)],
});

So, to me, it seems perfectly fine that fresh.gen.ts isn't populated. Have you seen the sample plugin used in Fresh's test case for this stuff?
If you don't notice anything immediately, then I can try checking out your project and debugging.

Well, curiosity got the best of me. I did check out the repo and fix the problem. You write

The route /postcss/global.css cannot be found (404)

But does this at all work without a plugin? I fixed the problem by moving the file to the static folder. Then there's no more 404 on the request for the stylesheet. Am I misunderstanding something?

Update - Fixed
I added a _404.tsx route and a log into the plugins route handler like this:
const postcssRoute = {
    path: "postcss/[...path]",
    handler: async (
      req: Request,
      ctx: HandlerContext,
    ): Promise<Response> => {
      logger.info(`Received Request for ${req.url}`);

The route seemingly never got called.
Reason - Broken Path in Plugin!
When you look at the Path postcss/[...path]  you see, that it misses a starting slash!
When I refactor it to /postcss/[...path], it works flawlessly!
Thoughts / Questions
Does a relative url make sense at this place?
Since plugins can only be added at the top level of a fresh application, I cannot tell what a relative url at this point is even supposed to do.
Maybe you have a valid usecase for this?
Otherwise I'd simply throw an error at the plugin developer for this :D

Sorry for the slow response. Just confirming that it all works fine now? A few followup items then:

A larger explanation about plugins. I'm thinking of an example page like "Writing a plugin".
Link to the test plugin from the plugin concept page. (I'm already doing this in the "plugins create islands" PR, but maybe we need to do this now, since it's easy, and that PR's changes won't get released until at least 1.4 is out.)
Some developer experience improvements about routes. There are actually two cases to think about:
3a) The plugin developer: are their routes defined correctly? (This is the part that would help you.)
3b) The plugin user: are the plugins clashing with other plugins or locally defined routes?

Hey @deer, no problem, since I solved my problem the follow ups are kinda lower prio.
Nice that you think about all these things!

Do you have a discussion page for how we should ease debugging of plugins for plugin developers and for the users in the end? I think I've some ideas to add to this discussion!

Can you send me the PR about the "Plugin creating Islands" thing? I'm interested in what it says.

Is there already a WIP for the "Writing a Plugin" page? If not, I'm interested in starting one at the week end or reviewing one.

I opened https://github.com/denoland/fresh/issues/1416 a few weeks ago. I started a branch and a guy continued with a PR that I haven't looked at yet. More discussion and ideas are always welcome.
https://github.com/denoland/fresh/pull/1472
I haven't started anything yet. I would happily review your PR if you create one!

@bjesuiter, can you close this? I don't think there's anything else to do here. Especially since https://github.com/denoland/fresh/pull/1594 has been merged, the path specification is now easier. If there's something else to do, please let me know.


extract.py: AttributeError: 'str' object has no attribute 'items'

Hi Rado,
when trying to update autocmake for mathlibs-tester I got this Python error in /cmake/autocmake/extract.py
milias@lxir074.gsi.de:~/Work/qch/software/small_projects/mathlibs-tester/cmake/.python -V
Python 2.7.3
milias@lxir074.gsi.de:~/Work/qch/software/small_projects/mathlibs-tester/cmake/.python update.py --self
- fetching autocmake/configure.py
- fetching autocmake/__init__.py
- fetching autocmake/external/docopt.py
- fetching autocmake/external/__init__.py
- fetching autocmake/generate.py
- fetching autocmake/extract.py
- fetching autocmake/interpolate.py
- fetching autocmake/parse_rst.py
- fetching autocmake/parse_yaml.py
- fetching update.py
milias@lxir074.gsi.de:~/Work/qch/software/small_projects/mathlibs-tester/cmake/.python update.py  ..
- parsing autocmake.yml
- assembling modules: [##############################] (15/15)Traceback (most recent call last):
  File "update.py", line 288, in <module>
    main(sys.argv)
  File "update.py", line 246, in main
    process_yaml(argv)
  File "update.py", line 163, in process_yaml
    modules, cleaned_config = fetch_modules(config, relative_path, download_directory)
  File "update.py", line 70, in fetch_modules
    d = to_d(v)
  File "/u/milias/Work/qch/software/small_projects/mathlibs-tester/cmake/autocmake/extract.py", line 23, in to_d
    for k, v in x.items():
AttributeError: 'str' object has no attribute 'items'
milias@lxir074.gsi.de:~/Work/qch/software/small_projects/mathlibs-tester/cmake/.

Please provide me more details on how to reproduce the error. Where can I find the repo? Which branch? And so on. Thanks!

Ah, right. This is derived project. Seeking how to reproduce it in autocmake ...

But where can I find it?

Hi,
it's this open project, https://github.com/miroi/mathlibs-tester/tree/update_autocmake
in which I wanted to upgrade autocmake.

You are missing the source: keys for plugins and custom.

So my autocmake.yml is wrong ? Could you please specify what parts are incorrect ? https://github.com/miroi/mathlibs-tester/blob/update_autocmake/cmake/autocmake.yml

You are missing a "source:" after https://github.com/miroi/mathlibs-tester/blob/update_autocmake/cmake/autocmake.yml#L37 and after https://github.com/miroi/mathlibs-tester/blob/update_autocmake/cmake/autocmake.yml#L47.

Works, thanks !
Closing this issue.


Cases in China

Suggestions by Cindy Chen in the Spreadsheet:
我发现北京卫健委每天都会以图标格式公布确诊患者的信息， 如果想做些关于效率的国别分析，这些数据都很clean的
http://wjw.beijing.gov.cn/xwzx_20031/wnxw/202002/t20200201_1622225.html
http://wjw.beijing.gov.cn/xwzx_20031/xwfb/202001/t20200131_1622025.html
上海卫建委也每天提供图表，只是关注的点不太一样
http://wsjkw.sh.gov.cn/xwfb/20200202/f5b6ce31678e43dca76c09d93ec4142b.html
合肥的报道也做的非常好，和你用的列都是一样的，我想这组数据可以用于做对照表http://www.bjnews.com.cn/news/2020/01/27/680324.html
If someone volunteers to add the data from Hefei, I can give you permission to edit the Spreadsheet.

There's an R package called "nCov2019" that records cases of each Province.

There's an R package called "nCov2019" that records cases of each Province.

Please provide a link in the future. Is this what you found? It is good to know. What we are doing (building a profile for each case) should be much more useful to understand the early epidemics.

@qingyuanzhao Yes that is what I found.
I can record the data from Hefei using the link above that records the 13 cases in Hefei up to midnight, Jan.26th.
http://www.bjnews.com.cn/news/2020/01/27/680324.html

@Elsa-Yang98
Can you give me your email? Also I will put you as an appointee to this issue (doesn't mean you need to do more beyond the 13 Hefei cases).

I just found these three links recording up to midnight Jan.30th of the 50 cases in Hefei. I haven't completed the gap between Case 13 and Case 39--that's between Jan.26th and Jan.29th.
http://www.cnr.cn/ah/news/20200130/t20200130_524953535.shtml
https://m.chinanews.com/wap/detail/zw/sh/2020/01-30/9073457.shtml
http://society.people.com.cn/n1/2020/0131/c1008-31565828.html
Sure, my email is<EMAIL_ADDRESS>

@Elsa-Yang98 I'm sending you an invitation to edit the Spreadsheet. Please record these cases using the "In China" sheet.

@qingyuanzhao  I finished the 13 cases in Hefei, please see if it's alright. I think the notes part should be written in English.... I will try to record all the 50 cases in Hefei

Looks perfect. I'll verify once I can get to it.

I finished all the 75 cases that are confirmed of the disease up to Feb.2nd
This Wechat official account updates info daily 合肥交通广播
https://mp.weixin.qq.com/s/uhsgv_QO8DiXPjCkanMsfg

南京‘健康南京’这个公众号统计的也很相信，每天更新
https://mp.weixin.qq.com/s/29lyOHW8lUhWrVZ-zpMxTg

Thanks, this data is very high-quality. Could you try to replicate the analysis here for the Hefei data? It'd be great to compare the estimates across different sources.
https://htmlpreview.github.io/?https://github.com/qingyuanzhao/2019-nCov-Data/blob/master/Feb3.html
Let me know if you need any help. The easiest way is to create a "fork" of this GitHub so you can work and update locally.

Nanjing
28-Jan: https://mp.weixin.qq.com/s/WaeUpQBuiJi_jPFhVFziQQ
29-Jan: https://mp.weixin.qq.com/s/q6oSShtpETmjdjMt06nlUw
30-Jan: https://mp.weixin.qq.com/s/BB_OX-wNCXz8GU0ZZYbjTA
31-Jan:
https://mp.weixin.qq.com/s/xjUZEj-T1iE2V6rc5BX-iQ
1-Feb: https://mp.weixin.qq.com/s/mDlDpN9hgnKRtXGGh2-W7w
2-Feb:
https://mp.weixin.qq.com/s/29lyOHW8lUhWrVZ-zpMxTg
3-Feb:
https://mp.weixin.qq.com/s/k4JsCyIwEbGjO82v4f5PAA
4-Feb:
https://mp.weixin.qq.com/s/USJ6P8Yfa7CuaNdGzvp_PQ
5-Feb: https://mp.weixin.qq.com/s/RIRb_rg4oaCJjAMCrk4HuQ

安徽阜阳
up to 27-Jan:
https://www.hotbak.net/key/阜阳23例确诊病例信息居住地全公布阜南县.html
28-Jan:
https://feng.ifeng.com/c/7teaNNMVwbG
29-Jan:
http://www.ahbbs.cc/wap/37141-1.html
30-Jan:
https://feng.ifeng.com/c/7thljdXtGyp
31-Jan:
https://www.fynews.net/article-188245-1.html
1-Feb:
https://www.fynews.net/article-188282-1.html
2-Feb:
http://ah.ifeng.com/a/20200203/8167754_0.shtml
3-Feb:
https://feng.ifeng.com/c/7toRODUgy80
4-Feb:
http://www.yxxsyxx.com/article-188422-1.html
5-Feb:
https://www.fynews.net/article-188480-1.html
6-Feb
http://www.ahwang.cn/newsflash/20200207/1978488.html

河南信阳：
Up till 31-Jan:
http://www.scxww.cn/xww/xwsd/8683.shtml contains almost(?) all Henan cases up till 31-Jan.
30-Jan:
https://dahebao.cn/news/1488767?cid=1488767
31-Jan:
http://www.scxww.cn/xww/xwsd/8575.shtml
1-Feb:
http://www.scxww.cn/xww/xwsd/8658.shtml
2-Feb:
https://m.thepaper.cn/newsDetail_forward_5776587
3-Feb:
https://4g.dahe.cn/news/20200204587310
4-Feb:
http://newwap.baoxiaofeng.com/news/73278
5-Feb:
https://www.twoeggz.com/info/216448.html

Anhui-Anqing:
You can find all the information here http://wjw.anqing.gov.cn/13971952.html
10-Feb: http://wjw.anqing.gov.cn/13971952/98351812.html
9-Feb: http://wjw.anqing.gov.cn/13971952/98211011.html
8-Feb: http://wjw.anqing.gov.cn/13971952/98087944.html

Anhui-Bengbu:
You can find all the information here: http://wjw.bengbu.gov.cn/
9-Feb: http://www.bbnews.cn/p/7471.html?spm=zm1143-<IP_ADDRESS>.bfnO3w
8-Feb: http://www.bbnews.cn/p/7450.html?spm=zm1143-<IP_ADDRESS>.QDW9PD
5-Feb: http://www.bbnews.cn/p/7413.html?spm=zm1143-<IP_ADDRESS>.vnxSjV

Tianjin
http://tj.bendibao.com/news/2020130/86602.shtm (being updated)


OTLP receiver: Convert start timestamps to Prometheus zero samples

Modify the OTLP receiver to, if the --created-timestamp-zero-ingestion feature flag is true, convert data point start timestamps to Prometheus zero samples 1 millisecond before the sample itself, iff the following is true: The OTel start timestamp is greater than zero and equal to the data point timestamp.
I'm also adding tests, copied from https://github.com/open-telemetry/opentelemetry-collector-contrib, with attribution.
Fixes #14600.

If reviewers prefer, I can factor out the added tests into another PR.

I rebased the PR with main changes that included some of the tests files also present here resulting in a much smaller PR for review 👍

Rebased the PR on main, and marked ready for review.

Hi, I believe things moved on a bit in the fork we are using to test this out.
For instance the "subtract 1 millisecond" turned out to be wrong.
So, this PR should be updated again from the fork.


AttributeError: 'ScalaMonitor' object has no attribute 'comm'

Summary of Issue
Note - Related container log error: ERROR JupyterSparkMonitorListener: Exception sending socket message:
Hello! We are running a Jupyterhub service that spawns Kubernetes pods for single user servers. We also run our jupyterhub proxy as a separate service. We've been experiencing a very persistent, but flaky, issue with the Sparkmonitor extension. When creating a spark session, I am seeing the following error AttributeError: 'ScalaMonitor' object has no attribute 'comm'. However, this does not happen consistently. I oftentimes am able to create a session where the monitor will work but then if I restart the kernel, the monitor is broken with the above error (or vice versa).
For example, on my first attempt at creating a spark session, the extension works fine. Container log:
INFO:SparkMonitorKernel:Starting Kernel Extension
INFO:SparkMonitorKernel:Socket Listening on port 59845
INFO:SparkMonitorKernel:Starting socket thread, going to accept
INFO:SparkMonitorKernel:59845
INFO:SparkMonitorKernel:Adding jar from /work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/listener_2.12.jar
[I 2021-12-21 23:45:52.892 SingleUserNotebookApp log:189] 101 GET /user/username/api/kernels/e5800417-4831-4bb4-9eec-85d7fb16b388/channels?session_id=aca66ed3-e383-4dfd-bd21-eae11e96ccb2<EMAIL_ADDRESS>1877.03ms
[I 2021-12-21 23:45:53.002 SingleUserNotebookApp log:189] 101 GET /user/username/api/kernels/e5800417-4831-4bb4-9eec-85d7fb16b388/channels?session_id=26868351-57cd-43c6-b14b-c85bd1842316<EMAIL_ADDRESS>2.18ms
[I 2021-12-21 23:45:53.147 SingleUserNotebookApp log:189] 101 GET /user/username/api/kernels/e5800417-4831-4bb4-9eec-85d7fb16b388/channels?session_id=0ccb7d08-e61b-44e6-a08c-57a36d2797ed<EMAIL_ADDRESS>2.12ms
[I 2021-12-21 23:45:55.711 SingleUserNotebookApp log:189] 200 GET /user/username/api/terminals?1640130355597<EMAIL_ADDRESS>1.13ms
[I 2021-12-21 23:45:56.253 SingleUserNotebookApp log:189] 200 GET /user/username/api/sessions?1640130356141<EMAIL_ADDRESS>1.09ms
[I 2021-12-21 23:45:56.254 SingleUserNotebookApp log:189] 200 GET /user/username/api/kernels?1640130356142<EMAIL_ADDRESS>1.01ms
INFO:SparkMonitorKernel:SparkMonitor comm opened from frontend.
INFO:SparkMonitorKernel:SparkMonitor comm opened from frontend.

To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

  FutureWarning)
[I 2021-12-21 23:46:39.217 SingleUserNotebookApp multikernelmanager:534] Kernel restarted: e5800417-4831-4bb4-9eec-85d7fb16b388

But if I restart the kernel and try again, the extension fails with AttributeError: 'ScalaMonitor' object has no attribute 'comm' (Note: it does not consistently follow this pattern of first time it works, second time it fails. Often the first session will fail and then it works after a kernel restart. Or it will fail three times in a row then work.)
INFO:SparkMonitorKernel:Client Connected ('<IP_ADDRESS>', 35472)
Exception in thread Thread-8:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/kernelextension.py", line 126, in run
    self.onrecv(msg)
  File "/work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/kernelextension.py", line 145, in onrecv
    'msg': msg
  File "/work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/kernelextension.py", line 223, in sendToFrontEnd
    monitor.send(msg)
  File "/work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/kernelextension.py", line 57, in send
    self.comm.send(msg)
AttributeError: 'ScalaMonitor' object has no attribute 'comm'

Container log on failure:
[I 2021-12-21 23:46:39.219 SingleUserNotebookApp log:189] 200 POST /user/username/api/kernels/e5800417-4831-4bb4-9eec-85d7fb16b388/restart?1640130391290<EMAIL_ADDRESS>7817.75ms
[I 2021-12-21 23:46:39.388 SingleUserNotebookApp log:189] 101 GET /user/username/api/kernels/e5800417-4831-4bb4-9eec-85d7fb16b388/channels?session_id=26868351-57cd-43c6-b14b-c85bd1842316<EMAIL_ADDRESS>2.70ms
INFO:SparkMonitorKernel:Starting Kernel Extension
INFO:SparkMonitorKernel:Socket Listening on port 53939
INFO:SparkMonitorKernel:Starting socket thread, going to accept
INFO:SparkMonitorKernel:53939
INFO:SparkMonitorKernel:Adding jar from /work/venv_core_ml_kernel/lib/python3.7/site-packages/sparkmonitor/listener_2.12.jar
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
I1221 23:46:45.923640   389 sched.cpp:232] Version: 1.7.2
I1221 23:46:45.925024   378 sched.cpp:336] New master detected at master@<IP_ADDRESS>:5050
I1221 23:46:45.925308   378 sched.cpp:401] Authenticating with master master@<IP_ADDRESS>:5050
I1221 23:46:45.925323   378 sched.cpp:408] Using default CRAM-MD5 authenticatee
I1221 23:46:45.925534   382 authenticatee.cpp:97] Initializing client SASL
I1221 23:46:45.926290   382 authenticatee.cpp:121] Creating new client SASL connection
W1221 23:46:45.928249   388 process.cpp:838] Failed to recv on socket 437 to peer '<IP_ADDRESS>:37450': Decoder error
I1221 23:46:45.929303   375 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1221 23:46:45.929327   375 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1221 23:46:45.929982   383 authenticatee.cpp:259] Received SASL authentication step
I1221 23:46:45.930640   386 authenticatee.cpp:299] Authentication success
I1221 23:46:45.930742   385 sched.cpp:513] Successfully authenticated with master master@<IP_ADDRESS>:5050
I1221 23:46:45.933359   377 sched.cpp:744] Framework registered with 195f7369-64e5-4bc5-a8ed-8b576994450a-11038
21/12/21 23:46:46 WARN MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
INFO:SparkMonitorKernel:Client Connected ('<IP_ADDRESS>', 35472)
INFO:SparkMonitorKernel:SparkMonitor comm opened from frontend.
[I 2021-12-21 23:46:49.219 SingleUserNotebookApp log:189] 200 GET /user/username/api/terminals?1640130409107<EMAIL_ADDRESS>1.17ms
[I 2021-12-21 23:46:49.220 SingleUserNotebookApp log:189] 200 GET /user/username/api/sessions?1640130409109<EMAIL_ADDRESS>1.04ms
[I 2021-12-21 23:46:49.222 SingleUserNotebookApp log:189] 200 GET /user/username/api/kernels?1640130409111<EMAIL_ADDRESS>0.99ms
21/12/21 23:46:51 ERROR JupyterSparkMonitorListener: Exception sending socket message:
java.net.SocketException: Broken pipe (Write failed)
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
        at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
        at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
        at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
        at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
        at sparkmonitor.listener.JupyterSparkMonitorListener.send(CustomListener.scala:54)
        at sparkmonitor.listener.JupyterSparkMonitorListener.onExecutorAdded(CustomListener.scala:652)
        at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:63)
        at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
        at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
        at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
        at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:115)
        at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:99)
        at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
        at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
        at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
        at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
        at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
        at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1319)
        at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)

I'm experiencing this issue even with a minimal configuration such as this:
from pyspark.sql import SparkSession
spark = SparkSession.builder\
        .config('spark.extraListeners', 'sparkmonitor.listener.JupyterSparkMonitorListener')\
        .config('spark.driver.extraClassPath', '/source/virtualenv_run_jupyter/lib/python3.7/site-packages/sparkmonitor/listener_2.12.jar:/opt/spark/extra_jars/guava-27.0.1-jre.jar')\
        .getOrCreate()

Additional Details
We are using the following Jupyter related packages/dependencies in our containers:
ipykernel==5.3.4
ipython==7.20.0
jupyter-client==6.1.5
jupyter-core==4.7.1
jupyter-packaging==0.10.2
jupyter-server==1.7.0
jupyter-server-proxy==1.6.0
jupyter-telemetry==0.1.0
jupyterhub==1.4.2
jupyterlab==3.0.16
jupyterlab-pygments==0.1.2
jupyterlab-server==2.5.2
jupyterlab-widgets==1.0.0
nbconvert==5.5.0
nbformat==5.1.3
traitlets==4.3.3

Any assistance with troubleshooting this issue would be greatly appreciated. A lot of our users really like the Sparkmonitor! Please let me know if I can provide any other details about our setup that would be helpful. Thank you!

Is there any update on this issue?
We have observed the same when using Jupyter Enterprise Gateway to spawn pods on Kubernetes for Jupyter Notebook kernels.
We were able to get around the issue for now by adding a sleep in the startConnection function of the JupyterSparkMonitorListener in CustomListener.scala file, before the Socket object is created

hi @krishnan-r
I think the issue is somewhere around the "server socket" handling that is happening in the "run()" method: https://github.com/swan-cern/sparkmonitor/blob/master/sparkmonitor/kernelextension.py#L103

A broken pipe error could be because the server / tcp listener socket closed the client connection and there is no logic on client side to catch this and re-establish the connection. (There is also a scope of improvement on scala client code)

I m still unclean so as to why the "comm" issue is happening.. I am planning to look into this further but if you have any pointer please let me know.
thanks!

Apologies for the late response, I'm able to reproduce this locally when doing Restart Kernel and Run All Cells... and I think it might be a race condition affecting other scenarios as well.
It's happening because the execute requests from the frontend reach the kernel before the frontend extension can connect to the kernel comm. This causes spark to start the listener in the JVM which sends messages on the socket to the kernel. The kernel side of the socket tries to forward the message to the frontend on a non-existing comm object, raises an exception and no longer reads the socket causing a broken pipe. (The code here could use some improvement for the error message)
One possible solution I think would be to make the kernel connect to the frontend via comm instead of the frontend to the kernel. This way we can block and ensure a connection before an execute request with spark application code is executed.
Or alternatively we need an extension point on the frontend which is earlier than notebookPanel.sessionContext.session.kernel.statusChanged to open a comm before other code can run.
This possibly never occurred in our deployment because we have an additional step on a UI where the user clicks a button to start the spark context, which only happens after a few seconds.
I will put together a draft PR with a fix soon

I've successfully resolved a same problem, and it's about browser cache!
The log of jupyterlab didn't show "comm opened from frontend" until I cleaned the browser cache!
INFO:SparkMonitorKernel:Starting Kernel Extension
INFO:SparkMonitorKernel:Socket Listening on port 33415
INFO:SparkMonitorKernel:Starting socket thread, going to accept
INFO:SparkMonitorKernel:33415
INFO:SparkMonitorKernel:Starting Kernel Extension
INFO:SparkMonitorKernel:Socket Listening on port 52331
INFO:SparkMonitorKernel:Starting socket thread, going to accept
INFO:SparkMonitorKernel:52331
INFO:SparkMonitorKernel:Starting Kernel Extension
INFO:SparkMonitorKernel:Socket Listening on port 45483
INFO:SparkMonitorKernel:Starting socket thread, going to accept
INFO:SparkMonitorKernel:45483
INFO:SparkMonitorKernel:Adding jar from /root/miniconda3/envs/py37/lib/python3.7/site-packages/sparkmonitor/listener_2.12.jar

After I cleaned the browser cache, jupyterlab log says comm opened, and the sparkmonitor panel shows up! 🎉
INFO:SparkMonitorKernel:SparkMonitor comm opened from frontend.

Hi, we also experience the same issue. Do you have any ETA on when it is going to be fixed?

Seeing this as well. Although I did not have to run the spark session creation quickly which led me to believe that it might not be a race condition (at least in the way I saw it).
In the console of the browser I see the following:
JupyterLab SparkMonitor is activated!
Plugin `jupyterlab_sparkmonitor` failed to activate
TypeError: Cannot use 'in' operator to reach for 'rank' in false
...
SparkMonitor: Starting Comm with kernel.
SparkMonitor: Starting Comm with kernel.
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'connect')

The second error is coming from here. I can't tell where the first one is coming from.
@diocas @etejedor Any idea what is going on here?

Downgrading to sparkmonitor==2.1.1 fixes issue on my end. After seeing the following the browser console, if is safe to start the spark session
SparkMonitor: Starting Comm with kerenel
SparkMonitor: Connection with comms established

Note: The race condition still exists, but the issue with the JS / TS plugin crashing is not an issue.

Oh wow, okay so this is more finky than I thought.

The browser console logs do not seem to be a reliable signal for the python kernel extension to actually be connected and have the comm attribute.
It seems, fairly consistently, that if I shutdown the kernel after connection has been established the front end will rail to re-connect.

There are definitely still some other oddities floating around, can't pin it down 100%.
Currently testing on jupyterlab==3.6.7 and sparkmonitor==2.1.1.
And sparkmonitor==3.0.2 still failing with issues mentioned above.

Here is some code to check if the comms have been established, at least this gives some user feedback and prevents the extension from crashing and being unusable afterwards.
import time
from sparkmonitor import kernelextension

TIMEOUT = 60
INC = 10

count = 0
while not hasattr(kernelextension.monitor, 'comm'):
    if count == 0:
        print("Waiting for sparkmonitor connections to be established...")
    elif count == 30:
        print("This is taking longer than usual... maybe try:")
        print("\t1.  Refresh your browser then re-run the cell.")

    if count >= TIMEOUT:
        raise RuntimeError("Sparkmonitor connections failed to establish!")

    time.sleep(INC)
    count += INC

print("Sparkmonitor connections established!")


Delete jdk_jdi_sanity from jdk_svc_sanity for duplicate testcases

jdk_svc_sanity and jdk_jdi has 32 duplicate testcases, which both include jdk_jdi_sanity. So delete jdk_jdi_sanity from jdk_svc_sanity to avoid duplicate testcases.
Fixes: #3993
Signed-off-by: sendaoYan<EMAIL_ADDRESS>

local test result:

test-jdk_svc_sanity.log


Fix #1183 - Display of validation errors lacks detail.

When we upgraded from PureScript 0.12 we lost automatically-generated
Show instances for parameterised types. That had the knock-on effect
of us creating a custom show function for ValidationError. But that
custom function didn't show enough detail.
I've fixed this by going back to our fork of purescript-bridge and
adding show support for parameterised types.


support pebble service manager

https://github.com/canonical/pebble

issue opened on sosreport to add support for pebble there - https://github.com/sosreport/sos/issues/3207


Urgent Bug Remove #228  gitignore update

Urgent Bug Remove

Proposed Changes
update .gitignore. As of now evry time user install npm every local machine has its own package-lock.json , when user push , it also goes in branch and it causes various issues. This is has been resolved now

Check List (Check all the applicable boxes)

[x] Bug(s) fixed
[ ] Enhancement done
[ ] New Feature(s) added
[ ] Documentation changed
[x] All new and existing tests passed.
[x] My code follows the code style of this project.

@thekavikumar  Solved Issue number #228 , Please check and merge this PR with proper labels of GSOC and level2 or level3


How can I get only data without column headers when i use  Invoke-Sqlcmd

I am trying to get column data and assign it to a variable which I want to later compare with a value of another variable using Invoke-Sqlcmd command. I am able to get data from the table but its coming with column header. Is there away how I can only get data without the column header? I have tried to use Select -ExpandProperty but I am getting errors. Below is the code and error message I am getting
$dbVersion =Invoke-Sqlcmd -server localhost -Database Test 'Select versionno from Employee where id =1'
PS SQLSERVER:> $dbVersion =$dbVersion | Select -ExpandProperty $dbVersion
Select : Property "System.Data.DataRow" cannot be found.
At line:1 char:26
$dbVersion =$dbVersion | Select -ExpandProperty $dbVersion
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo : InvalidArgument: (System.Data.DataRow:PSObject) [Select-Object], PSArgumentException
FullyQualifiedErrorId : ExpandPropertyNotFound,Microsoft.PowerShell.Commands.SelectObjectCommand

I have managed to get only data in the column without column headers. Below is the final fine code
PS SQLSERVER:> $Version =Get-Content -Path C:\Users\Admin\Documents\MSSQLScripts\Db-Version.txt
PS SQLSERVER:> $dbVersion =Invoke-Sqlcmd -server localhost -Database Test 'Select versionno from Employee where id =1'
PS SQLSERVER:> $dbVersion =$dbVersion | Select -ExpandProperty versionno
In this case Versionno is the column header in the db and I have supplied it with the Select -ExpandProperty command.
That solved my problem.


signal handlers: check for number of arguments gets fooled by **kwargs handler

I am trying to register one generic handler to a couple of DBus signals with different signatures. This should be not a big deal in the world of duck typing - just use **kwargs in your handler and see what number of args you get.
Unfortunately, dbus-next is stopping me here: https://github.com/altdesktop/python-dbus-next/blob/04e7c0428ec2e4020d4b95aa70e10feb8e53e22e/dbus_next/proxy_object.py#L105
This check comes to the conclusion that my handler has one argument (**kwargs), but my signal has two.
Is there any chance to improve this check, opt out of it or remove it completely? I think this static understanding of the number of arguments is not very Pythonic.

A keyword argument needs a keyword for naming, where would you expect them to come from ?


UWP "data" property on Action.Submit isn't accessible (data values not returned on action click)

There's no Data property on the AdaptiveSubmitAction. The event you get back for a button being clicked is called "Inputs", which would imply that it would NOT contain the Data key/value pairs (and today it does NOT).
Hence exposing the Data property on AdaptiveSubmitAction would allow the host to merge Inputs and Data into a single ValueSet. Plus having the property accessible could be useful for other reasons.

I might play around with adding the Data property to the object model if no one is currently working on this. Wanted to get toast activation in the Visualizer 100% wired up given that inputs are now returned :)

Go for it!


how to use param spin_dataloaders in Trainer

If I start training with 4 nodes, each equipped with 8 GPUs, and then save checkpoints, I later resume training from the saved checkpoint using spin_dataloaders=True. However, if I increase the nodes to 8 and continue training from the previous checkpoint, the data loader will be adjusted to include the previously seen data points. Should I need to manually skip the data points that were already processed? Will be the data loader will manage the distribution of data across the increased number of nodes for me?

Hi @HunterKruger, If you are using the Composer and Streaming, which I think you are, then you don't have to do anything explicitly. The dataset meta data will get saved when saving a checkpoints and it will load the same meta-data while loading the same checkpoints when you execute another run with autoresume=True and streaming dataset will skip the samples that is being already seen by your previous run. Streaming dataset also takes care of distributing the unseen samples (not been processed by the model) into nodes/gpus/workers.


Allow the wrong quotes to be used when there is an embedded quote in a string

I like enforcing single quotes most of the time (with quotmark: single) but it'd be nice to have an exception for strings that contain an embedded single quote (and no embedded double quote), i.e. allow "I'm pedantic" or 'He said "no"' even if quotmark would normally disallow this.

The quotmark option will be removed, you should consider using JSCS.


Lags when scrolling manga chapters

Steps to reproduce
1.Go to manga
2.Scroll chapters
Expected behavior
Scrolls smoothly without lags
Actual behavior
Lags even scrolling in slow manner
Crash logs
No response
Neko version
2.10.13
Android version
Android 12
Device
Oppo A74
Other details
I am not a troll, i always prefer this fork due to it's functionality. I know the work was hard but i post it so the devs could work on the problem right away
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] I have tried the troubleshooting guide.
[X] I have updated the app to the newest version Latest.
[X] I have filled out all of the requested information in this form.

Most performance issues with this screen have been resolved, can you attach a video of the lag

I also have observed this bug while trying to scroll up manga chapters. I'm using version 2.12.8 Android 11 Samsung Galaxy A70
https://user-images.githubusercontent.com/46072236/218318199-f57c1e9e-1787-4dff-baed-dd17868b6b18.mp4

I am also getting this and it's very annoying. It's almost impossible to scroll to the top of a chapter list as when you get close to the top it just teleports you down a chunk.
This also happens when scrolling quickly
I wouldn't call this lag, more likely a bug with the page

Any solution to this yet?
I use neko daily to read manga but this bug is making it horribly annoying to actually use the app properly. I didn't use to get this but suddenly about a few months ago this started happening. I've tried completely resting the app and reinstalling it, with no luck.

I have this issue as well.
It seems that when you scroll down enough on a manga's page so that the chapter lost covers the whole screen, scrolling back up becomes insanely difficult as it constantly bounces back down as you scroll.
This is the biggest issue I have with the app, everything else is pretty minor.

Here's a video that shows the issue. You can see that I really struggle to scroll back to the top until I flick the screen pretty hard.
https://github.com/CarlosEsco/Neko/assets/15630367/0dede21d-b074-4d81-8510-f2610a637e46


CUDA back-end not available even though toolkit is installed and setup.py was ran with no parameters.

I can import cudarray after installing everything but for some reason, I still can't use the CUDA back-end I know I have. Any help?

Ok so if I manually switch to the CUDA back-end, it says it can't find a file called libcudart.sc

Hi, have you seen this: http://andersbll.github.io/deeppy-website/installation-guide.html#verify-cuda-back-end ?
The error you post above doesn't seem like an error to me, so the installation of libcudarray might be fine. Have you remembered to update LD_LIBRARY_PATH to point to where libcudarray is installed?

I'm pointing to that but it's saying that it can't find libcudart.sc.7.5. How do I specify this?

You may also need to add the path to the cuda libraries to LD_LIBRARY_PATH.

How come I am getting this while others aren't?
On Sun, Feb 21, 2016 at 4:36 AM, Anders Boesen Lindbo Larsen <
<EMAIL_ADDRESS>wrote:

You may also need to add the path to the cuda libraries to LD_LIBRARY_PATH.
—
Reply to this email directly or view it on GitHub
https://github.com/andersbll/cudarray/issues/39#issuecomment-186793081.

Don't know, I'm not even sure what your precise error message is.

When I run the python  repl in the Shell and type "import cudarray", I get an error saying that it is unable to find the file libcudart.sc.7.5
Hie do I fix this error

and by libcudart.sc.7.5 you mean libcudart.so.7.5? If you are on Linux, then try running ldd <path to libcudarray.so in the command line. If it cannot locate libcudart.so.7.5, it probably means that your LD_LIBRARY_PATH doesn't contain the right path to your CUDA libraries.

That solved it. Thank you. I was at this for hours.

Actually there is un

Good to hear! :)

@Vatyx what command did you end up writing to get this to work? I'm on a Mac, and have tried
echo 'export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH' >> ~/.bashrc as well as
echo 'export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH' >> ~/.bash_profile
and have not been able to get rid of the error that you mention.

Are you instaling with sudo? In that case make sure the environment variables are also set for the root user.

solved it following these steps. Looks like Mac users on El Capitan need to disable SIP to change the library path.
I have run into yet another issue that has me at a loss. I think this should be the final issue before I'm up and running. In this block:
if _backend == 'cuda':     try:         from .cudarray import *         from .base import *         from .linalg import *         from .elementwise import *         from .reduction import *         from . import random         from . import nnet         from . import batch         from . import extra         wrap.cudart.initialize(_gpu_id)     except:         if _force_backend:             print('CUDArray: Failed to load CUDA back-end.')             raise         else:             print('CUDArray: CUDA back-end not available, using NumPy.')             # Try NumPy instead             _backend = 'bumpy'
I'm getting this error for each of the last 4 imports:
ImportError: cannot import name random
What is the from . import * doing? Can i replace the period with something that tells it exactly where to extract the file?

Great job solving the previous issue. Regarding the new problem: What version of Python version are you using? It seems like a problem with circular dependent imports (which is strange since I do not have that problem).

How to install windows


Cast string values to numeric when aggregating

Problem
Prisma's numerical aggregations works fine, but sometimes applications need to store numbers as strings in the databases, and the SDK doesn't allow to aggregate them.
It would be great to have a way to tell prisma to cast strings to numbers in order to be able to sum and do other aggregation operations.
Suggested solution
It is possible to do such thing manually in Posgres with raw SQL request like this:
 select sum(cast(amount AS numeric))  from transactions

You can use a raw query as a workaround here for now.

You can use a raw query as a workaround here for now.

You can use a raw query as a workaround here for now.

I already use a raw query with the example I gave. This is a feature request :)

You can use a raw query as a workaround here for now.

I already use a raw query with the example I gave. This is a feature request :)


Why is Result not utilized?

Why is it that get_* functions in Buf panic if there is no more data remaining, rather than returning an error? I wrote an extension trait for this that didn't take much time but it confuses me why it's preferable for a bad read to bring down its thread rather than allowing the user to handle it gracefully.

Because Result is intended to represent runtime errors that can happen in bug free code. Panics are used to represent errors due to bugs.

Do you disagree with the guideline?

I find that using Result would be a lot less opinionated about how errors should be managed, because it gives the option for graceful handling while also allowing a panic if you're sure it's acting as it's supposed to


Make server launch with no USB devices connected

Server/GUI will not launch if there is no Arduino connected or if it doesn't recognise a boardPort
Recommend that GUI launches but displays a message advising no USB device detected (or a dropdown to select a boardPort?)
npm run start:server

Returns:
1707633642050 Error Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02
node:events:508
    const err = new ERR_UNHANDLED_ERROR(stringifiedEr);
                ^

Error [ERR_UNHANDLED_ERROR]: Unhandled error. ({
  type: 'error',
  timestamp:<PHONE_NUMBER>050,
  class: 'Error',
  message: 'Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02',
  data: null
})
    at Board.emit (node:events:508:17)
    at Board.log (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:637:8)
    at Board.<computed> [as error] (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:648:14)
    at Board.finalizeAndBroadcast (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:363:12)
    at /Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:147:18
    at SerialPort.<anonymous> (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/firmata-io/lib/firmata.js:613:9)
    at SerialPort.emit (node:events:519:28)
    at SerialPort.emit (node:domain:488:12)
    at SerialPort._error (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/@serialport/stream/lib/index.js:200:10)
    at /Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/@serialport/stream/lib/index.js:242:12 {
  code: 'ERR_UNHANDLED_ERROR',
  context: {
    type: 'error',
    timestamp:<PHONE_NUMBER>050,
    class: 'Error',
    message: 'Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02',
    data: null
  }
}

We may be able to avoid having to input boardPort into server.ts before building it if we can implement a dropdown that includes all USB devices using WebUSB API:

https://developer.mozilla.org/en-US/docs/Web/API/WebUSB_API

We may be able to avoid having to input boardPort into server.ts before building it if we can implement a dropdown that includes all USB devices using WebUSB API:
* https://developer.mozilla.org/en-US/docs/Web/API/WebUSB_API

Currently the WebUSB API is in experimental phase and features that would be required to listen for USBConnectionEvents lacks support for common browsers such as Firefox and Safari. In saying that, from the looks of it, this API might potentially reduce the need for johnny-five. I'm happy to create a separate branch to play with it - but again it would be experimental only c:

Server/GUI will not launch if there is no Arduino connected or if it doesn't recognise a boardPort
Recommend that GUI launches but displays a message advising no USB device detected (or a dropdown to select a boardPort?)
npm run start:server

Returns:
1707633642050 Error Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02
node:events:508
    const err = new ERR_UNHANDLED_ERROR(stringifiedEr);
                ^

Error [ERR_UNHANDLED_ERROR]: Unhandled error. ({
  type: 'error',
  timestamp:<PHONE_NUMBER>050,
  class: 'Error',
  message: 'Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02',
  data: null
})
    at Board.emit (node:events:508:17)
    at Board.log (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:637:8)
    at Board.<computed> [as error] (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:648:14)
    at Board.finalizeAndBroadcast (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:363:12)
    at /Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/johnny-five/lib/board.js:147:18
    at SerialPort.<anonymous> (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/firmata-io/lib/firmata.js:613:9)
    at SerialPort.emit (node:events:519:28)
    at SerialPort.emit (node:domain:488:12)
    at SerialPort._error (/Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/@serialport/stream/lib/index.js:200:10)
    at /Users/osx/Desktop/FPV-RC/rc-fpv/server/node_modules/@serialport/stream/lib/index.js:242:12 {
  code: 'ERR_UNHANDLED_ERROR',
  context: {
    type: 'error',
    timestamp:<PHONE_NUMBER>050,
    class: 'Error',
    message: 'Error: No such file or directory, cannot open /dev/tty.usbmodemDC5475CBEFC02',
    data: null
  }
}

I'm not sure if it is possible to get a drop down list of usb devices that are not audio and video. I can try to find a work around and also try to implement a way to manually enter the config in the web gui instead of having to rebuild the server.


feat(update): allow the client to retrieve the document _source in updates

GetResult was present in the UpdateResponse, but no way was provided to set _source to true, needed to get that result (docs). The result also is not a full GetResult, containing only found and _source in my ES5 instance, so I added a separate struct for that.

Thanks for your PR.
This needs to be done via FetchSourceContext, which isn't added to the Update API. I'll prepare an update to 5.0.42.

ES also returns a full GetResult (which might miss some fields), so there's no additional struct necessary.

Ah, great! Somehow I totally missed that when grepping for _source in the update. I might have been on the master branch instead of v5. Thanks 😄


Add support for Realtime Database Transactions

Closes #108
See docs/realtime-database.rst for documentation and usage examples.
Todo

[x] Catch and convert the "Precondition failed" request exception
[x] Add support for deletions
[x] Revisit snapshotting
[x] Revisit naming
[x] Add documentation

:octocat:

Merged with 86b9dc0ce1672a067d3acfe1e0c779cd1e5bae52


[Poll] Do you use next-, prev- and close-dialog? (^V, ^G, ^X)

In 2008 Newsbeuter gained a new feature: users could now freely switch between
dialogs. For example, one could open save dialog and then quickly peek back at
the article to come up with a better filename. Or one could take a moment
while reading an article to go update the feeds to see if something more
important came up.
This is a nice feature, but it adds complexity to the code. I wonder if anyone
is really using it; maybe we can just get rid of it?
Just "yes" or "no" will be sufficient—I'm mostly interested in numbers here.

No.
I never used this feature and testing it now, I don't see a clear reason with it.
Neither "close-dialog" nor "^X" is anywhere to be found in the help-menu.

Neither "close-dialog" nor "^X" is anywhere to be found in the help-menu.

Indeed! Thanks for catching that; if we decide to keep the feature, I'll open an issue about this and fix it.

Yes
I mostly use it when listening to a podcast, I find it useful to have the show notes/sources open in one dialogue and go back to the feedlist to do something else while still being able to easily switch back if I hear something I'm interested in.

No. I primarily use Newsbeuter to store my headlines, that I'll read on my web browser. I don't use the dialog-navigation functionality.

I'm going to close the poll on February 1st (i.e. in a little over a week). Please vote if you didn't yet!

No. I don't use them because I don't need these features.

No, I just navigate the tree with open/close.

No, I almost always open my "unread articles" query feed and then walk through all the articles with the next-unread function and finally close the query feed again.

Hi everyone,
The poll is now over. To recap, the question was: is anyone using next-, prev- and close-dialog (^V, ^G, and ^X keys)?
We've had 9 participants (7 here in the issue tracker, 2 on the mailing list). One didn't explicitly vote but said that he wasn't aware of the features; this means he wasn't using them, which is good enough for the purposes of this poll. Another didn't explicitly vote but gave :+1: to a comment voting "no"; I'll have to assume this means "me too", i.e. "no".
Just one person voted "yes". This person also described his use-case: keeping the shownotes accessible while listening to a podcast. I like it.
Thus my decision is to keep these features.
Thank you to everyone who participated!
@flnds, the issue I promised to open is here: #490.


Doc outdated? Growing DB size on Windows now possible, since LMDB now uses native NT API?

(Bug is not the right category, it's more about the docs.)
I have a question about this:
https://docs.meilisearch.com/reference/features/configuration.html#options

MEILI_MAX_MDB_SIZE: The maximum size, in bytes, of the main database. The main database stores the processed data.
Depending on the OS, it is either the size that will be allocated on launch or the maximum size the database can attain.
On Windows it is a fixed size that will be allocated on launch. Because this allocates 100GiB on MeiliSearch launch, a Windows user can use this option to decrease the size of the database.

Does this mean that it's impossible to start with a small db size and let it grow as needed on windows? (By setting MEILI_MAX_MDB_SIZE to a small value during init, and a higher value later.)
I'm asking because I'd like to include MeiliSearch with my application just like sqlite (because I need a faster fuzzy search). And allocating a 100 GB file right away shouldn't be done in this case. It should grow as needed.
Is that possible?
Regarding LMDB's limitations, I found this:
https://stackoverflow.com/questions/33508305/lmdb-maximum-size-of-the-database-for-windows#comment62219239_33571804
Which links to this (now fixed) ticket: https://bugs.openldap.org/show_bug.cgi?id=8324
Does MeiliSearch take advantage of this fix? If so, what's the right way to start with the smallest db file possible and grow it as needed (on windows)? :)

Hey @Boscop !
We are aware of this fix, but it still hasn't been propagated to Meilisearch. Meilisearch depends on Heed as a wrapper for LMDB and Heed itself depends on mozilla bindings to the c library, that are outdated. We have plans to roll out our own bindings to LMDB sooner or later, but until then this restriction is still present... Until then, on windows, the only thing you can do is grow the db size manually unfortunately.

Hello @Boscop!
Marin explained it well!
More detail here if yo are interested: https://github.com/meilisearch/MeiliSearch/issues/851#issuecomment-820486885

@MarinPostma Thanks! But what is a good size to start with, and how do I know when it's necessary to grow it?
And for growing it, should I just increase the value of that env var?

@Boscop, a good size to start with will really depend on the amount of data you have, and your settings. I would go with whatever you can reasonably allocate, and see what's the size of your data using the /stats route. If the you see that you are using much less data that you allocated, then allocate a smaller db. You can also probably reduce the size of the update database (--max-udb-size) to something much lower, since you probably don't need that much space anyways.
Ultimately, if windows is your development environment, I suggest you run MeiliSearch in the WSL to avoid all those issue altogether.

@MarinPostma Thanks! But it only works in WSL2, not WSL1, right?
And I plan to bundle meilisearch.exe with my application just like sqlite (because I need a faster fuzzy search) so it should be portable & standalone, not require WSL/docker..
If I insert too many documents into the db so that the allocated size is not enough, what kind of error would it return?

I have never used WSL personally, so I can't give you an answer....
You will get an error like this one:
[
    {
        "status": "failed",
        "updateId": 0,
        "type": {
            "name": "DocumentsAddition",
            "number": 19547
        },
        "error": "heed error; MDB_MAP_FULL: Environment mapsize limit reached",
        "errorType": "internal_error",
        "errorCode": "internal",
        "errorLink": "https://docs.meilisearch.com/errors#internal",
        "duration": 0.043656761,
        "enqueuedAt": "2021-04-27T13:49:55.999377511Z",
        "processedAt": "2021-04-27T13:49:56.249900055Z"
    }
]

(the actual error message is subject to change)

Hello @Boscop, we released a new version of the search engine yesterday and we finally forked and updated LMDB.
We are now able to run our tests on windows :tada:
If you are still following the project and try this new release please let us know if you encountered any issues.

I close this issue then, feel free to re-open the issue if there are any issues @Boscop! And thanks again for your report!


Remove fail-fast matrix strategy from GitHub Actions

Description
Currently, every GitHub Actions workflow contains the fail-fast matrix strategy. However, none of these workflows are actually utilizing a matrix. Further, the on demand workflow has an improper indentation of this strategy block that causes a syntactical error.
This change remove the no-op strategy altogether to fix the broken on demand workflow and remove the IDE warnings regarding the missing matrix declaration.
Related issue and scope

[ ] I opened an issue to propose and discuss this change (#????)

My changes affect the following components

[ ] API
[ ] Controller
[ ] Message Bus (e.g., Kafka)
[ ] Loadbalancer
[ ] Scheduler
[ ] Invoker
[ ] Intrinsic actions (e.g., sequences, conductors)
[ ] Data stores (e.g., CouchDB)
[ ] Tests
[ ] Deployment
[ ] CLI
[x] General tooling
[ ] Documentation

Types of changes

[ ] Bug fix (generally a non-breaking change which closes an issue).
[ ] Enhancement or new feature (adds new functionality).
[ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

[x] I signed an Apache CLA.
[x] I reviewed the style guides and followed the recommendations (Travis CI will check :).
[ ] I added tests to cover my changes.
[ ] My changes require further changes to the documentation.
[ ] I updated the documentation where necessary.

Codecov Report

Merging #5421 (e643007) into master (88156c3) will decrease coverage by 72.34%.
The diff coverage is n/a.

:exclamation: Current head e643007 differs from pull request most recent head f657a87. Consider uploading reports for the commit f657a87 to get more accurate results

@@            Coverage Diff             @@
##           master   #5421       +/-   ##
==========================================
- Coverage   76.84%   4.51%   -72.34%
==========================================
  Files         241     241
  Lines       14630   14630
  Branches      616     616
==========================================
- Hits        11243     660    -10583
- Misses       3387   13970    +10583

see 206 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

From the linked documentation on fail-fast:

jobs.<job_id>.strategy.fail-fast applies to the entire matrix. If jobs.<job_id>.strategy.fail-fast is set to true, GitHub will cancel all in-progress and queued jobs in the matrix if any job in the matrix fails. This property defaults to true.

This strategy block is specifically used to create a matrix strategy in order to run a job multiple times with different parameters. For example, you may do something like this to run a job against multiple operating systems:
strategy:
  fail-fast: true
  matrix:
    os: [ubuntu-latest, windows-latest]

In this case, if say ubuntu-latest were to fail, then windows would be cancelled. But since you are not using a matrix strategy like this, the fail-fast would be a no-op.
I also get this warning in my IDE, although I do not know where to locate the schema it is using for this validation:

@clayburn Got it.
Thank you for the explanation.
I confirmed a build with the current master immediately stops on an error.
https://github.com/apache/openwhisk/pull/5423


Update wp-stock-information.md

Adjusted the name of the label in the documenation. The label is "Automatic Refresh" as described in the en-us.js file: https://github.com/SharePoint/sp-starter-kit/blob/master/solution/src/webparts/stockInformation/loc/en-us.js
Category

[x] Bug Fix
[ ] New Feature
Related issues: fixes #X, partially #Y, mentioned in #Z

thx Dennis, appreciated.


Wiegand reader

Hi, i would like to know if the ESP-Rfid is working with any reader in the Wiegand protocol? With this reader will work: https://www.zkteco.eu/index.php/categories-11/access-control-systems/rfid-external-readers/wiegand-reader

Yea, it's working pretty well with Wiegand readers. As well using similar reader with 125KHz/13.56MHz was bought from aliexpress (http://ali.onl/109k)


chore: Upgrade Orchestrator Lib packages

Description
closes #5095
Update Orchestrator Lib to 4.11.0-beta.20201203.cd29e25.
This change allows Composer on OS X to start without having to manually install icu4c first, Orchestrator now bundles this dependency in the lib.
Disable npmRebuild feature in electron-builder as well - the native rebuild has no benefit since Orchestrator's node packages run properly on our Electron version, and additionally causes a linking issue for Orchestrator on Mac.  Only Orchestrator is affected by this flag - no other libraries in Composer are affected by npmRebuild
Tested for regressions by running electron build on Windows 10, Mac OSX 10.15 and Ubuntu 20.04.  On Mac, the app is now bootable without icu4c being installed first.

Coverage remained the same at 55.807% when pulling 120f67e3c6ebb27328707cc0507e1db11aa5b9c8 on tachou/UpdateOrchestratorLibs into 0b71db2df5351e24ad07f2bba53acfedcb24c4ae on main.

@lei9444  can you confirm this with your mac?

I have tried the installer in mac, I can open it and update the bot, but when I start bot, composer crashed. But I think it is the m1 cpu cause the crashed issue

I have tried the installer in mac, I can open it and update the bot, but when I start bot, composer crashed. But I think it is the m1 cpu cause the crashed issue

I got a regular Mac and tried the same issue, app is opening fine, and editing is good, but the moment i click start, my app crashed.  @taicchoumsft i think we need to onhold on this.
Also, i found that, orchestrator seems to not allowing me to train an empty lu, some weird errors happens if i just switch everything from ToDoLuisSample to orchestrator.

I tested on my intel iMac, without pre-install icu4c, it works.

OK, we confirmed this would fix the issue using Zhixiang's env, which represents the most common case. The reason this fix wasn't working for me is my icu4c version is too low and in Leilei's case, he has the M1 cpu.


[feat] customize the storage path for data backup

Describe the problem
I hope to be able to customize the storage path for data backup.
Describe the solution you'd like
I hope to be able to customize the storage path for data backup.
Alternatives considered
No response
Additional context
No response

@qiuhaian It will increase in the future

#506 已经支持了自定义数据的存储路径，对于备份数据的存放路径是有什么考虑吗？临时放在下载中应该是够用了！


test: starkli declare workflow in test

Create integration test with mock starknet node where beerus requests are going to simulate starkli declare workflow function calls:

starknet_chainId
starknet_getClass
starknet_chainId
starknet_getNonce
starknet_specVersion
starknet_estimateFee
starknet_addDeclareTransaction

It has been decided that mock tests are redundant.


Microsoft.Compute and Microsoft.Storage resource providers

Following the tutorial I had to register Microsoft.Compute and Microsoft.Storage as resource providers.
I used the following commands:
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 93aa980e-1970-235c-228a-7b19880171b8
Version Independent ID: 6f7937a4-12e0-d61c-3bd8-c6347ec48385
Content: Tutorial - Create an Azure Red Hat OpenShift 4 cluster
Content Source: articles/openshift/tutorial-create-cluster.md
Service: container-service
GitHub Login: @sakthi-vetrivel
Microsoft Alias: suvetriv

@marceloazuma
Thanks for the feedback! We are currently investigating and will get back on this.

The document is updated, changes will go live soon.


Add action sections to scorecard page linking to resources and national map pages

Description
Addresses: Add share / additional resources section to scorecard page and Add Actions to Take section
New
Adds ActionSection component that we need for the scorecard and error page. Includes a header, subheader and CTA button.
Changed
Moved footer out of landing page into all pages. The map page needs to be made larger to fit the screen in a separate PR.
Testing and Reviewing
See screenshots

Does this cover the explanation of the score section? looks like it in the screenshot

screenshot

I just refreshed the text based on mocks but that section was there before


Thanks for this project

Just wanted to say thank you for making this project available.
I'm currently using it to do some interesting things with the telegraf binary && making service bindings available within telegraf.conf as environment variables.
https://github.com/aegershman/telegraf-on-cloudfoundry
That's all, just saying thanks

You're welcome @aegershman! I'm glad it helped and thanks for the nice feedback


have a story for pagination

At the least, there should be an example showing how to do it. It is not clear to me if we would want to make any changes to the core pggen library though.

Since writing this, I've implemented pagination using pggen, and it is a bit of a pain to do full bidirectional pagination with a static SQL query. Dynamic sql metaprogramming really is the most natural way to do this. We could provide some custom functionality in the same way that we handle updates and upserts, but I'm a bit concerned with that approach since there are so many different ways to do pagination.


chore(main): release 1.0.7

:robot: I have created a release beep boop
1.0.7 (2023-01-25)
update

更新首页列表获取数据 (181108c)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/School-of-Website-Engineering/Heimatoutiao/releases/tag/v1.0.7 :sunflower:


#99, #243, fixes for nested page back navigation, prevent infinity loop

Fix for navigating back from nested page (there is an issue when disconnecting handler), switched to look into loaded and unloaded events
Possible fix for #99 thanks to istanko comment and #243
Prevent infinity loop for handling focus and add the ability to set autofocus interval.
I also faced #239 , I was not able to even open a solution, I noticed that it is starting to "work" by commenting on this line in SampleApp.XF.iOS.csporj but not sure how to fix it for the XF iOS app...
<Import Project="$(MSBuildExtensionsPath)\Xamarin\iOS\Xamarin.iOS.CSharp.targets" />

I would upvote this for the possibility to change the auto focus interval alone.


Notification doesn't cancel for non connected calls

[x] I have read the Getting Started and Getting Started(with call invitation)
[x] I have already searched for the same problem

Environment

Technology
Version

Flutter version
3.22.2

Plugin version
^4.12.1

Android version
14

iOS version
17.4

macOS version

Xcode version
15.4

Google Chrome version
125.0.6422.142

Device information: Realme 11 Pro Plus - Android 14 and iOS 12
Description
Expected behavior:
Scheduled notifications should be canceled for call disconnection and non-connection. The recipients get calling notifications 15-20 mins after call disconnection (if the recipient goes back to the internet).
Current behavior:
The caller’s call gets disconnected after a minute if the callee has no internet network or is offline. Now, when the callee receives a call notification on the network/internet (back to internet) for the same call.
Steps to reproduce
Try with normal example repo.

I tested it and didn't find this problem
step:

b broke network
a send call invitation to b
a auto cancel the call invitation after timeout
b connect network

@yoer Please check your email. I have shared a video for this issue.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v5.13 to before v5.13.4",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=01e055c120a46e78650b5f903088badbbdaae9ad",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ccdf7e073170886bc370c613e269de610a794c4a"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "01e055c120a46e78650b5f903088badbbdaae9ad",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "ccdf7e073170886bc370c613e269de610a794c4a",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "arch_topology: Avoid use-after-free for scale_freq_data\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.13.4 by commit ccdf7e073170886bc370c613e269de610a794c4a, it was introduced in version v5.13 by commit 01e055c120a46e78650b5f903088badbbdaae9ad. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001187


Invalid volume specification

Unfortunately, I've been struggling to get my head around niche docker issues.
When initially running run-docker.sh, everything is functional, but I then get a docker error as illustrated below.
Given it then mentions the path to the Vivado installation, I'm not sure what volume specification I'm meant to replace that with.

Hello,
Prior to running run-docker.sh, what is the value of your VIVADO_PATH environment variable? Based on the above it looks like it contains two copies of the same path whereas it should be export VIVADO_PATH=/tools/Xilinx/Vivado/2019.1

I'm closing this issue due to inactivity. Please feel free to reopen, or discuss further on the FINN gitter channel on https://gitter.im/xilinx-finn/community


Slicer log should not need to be opened

@thenineteen, I see you keep opening the log in Slicer to check your printed stuff. If that information is important, it should be returned by the functions in mega_analysis, not just printed out. The info can then be added to the module GUI.

so e.g. exclude_postictals(df) is a function which needs to remind the user that postictals are excluded.
if the below is true
if POST_ictals:

then this was called by top level MEGA ANALYSIS and all the data for all semiology will be without postictals.
If instead exclude_postictals(df) is called as the other exclusions at the end of exclusions.py are called for excluding priors, then GUI should print 'Excluded post-ictal semiology in specific query' ideally next to the semiology checkbox,

^ how can we return these to your API? I don't want to say return as it will stop executing the rest

Why does the user need to be reminded of something they checked? I'm not sure I'm following. Can't this exclusion be treated like the others?

I think there a few things the user wants to know that should not be in sclier logging but printed in the GUI:

the numbers of patients and studies included in the entire database when the data is loaded
the numbers of lat and loc data per each query - #55 addresses this one
exclusions. But if you think this is too much, then post-ictal exclusions as a minimum should be printed in the GUI as these are semiology specific. The alternative to this would be to have entirely new semiology e.g. "POST ICTAL APASHIA" which when ticked, automatically ONLY includes postictals (would have to write the code to do this) and when "APHASIA" semiology is clicked, only those that don't have post-ictal are included.

the numbers of patients and studies included in the entire database when the data is loaded

Then those values should be returned by whichever function is printing them.

the numbers of lat and loc data per each query - #55 addresses this one

Same as 1.

exclusions. But if you think this is too much, then post-ictal exclusions as a minimum should be printed in the GUI as these are semiology specific. The alternative to this would be to have entirely new semiology e.g. "POST ICTAL APASHIA" which when ticked, automatically ONLY includes postictals (would have to write the code to do this) and when "APHASIA" semiology is clicked, only those that don't have post-ictal are included.

I don't understand this point. We can discuss this in the next meeting.

Any updates on this? If mega_analysis returns this information, I'll be able to show it on Slicer.

mega_analysis or MEGA_ANALYSIS.py? I can get these to be added to the return, but then we would need to update all calls to these functions
e.g. MEGA_ANALYSIS currently returns three dataframes, I can add a few numbers to this like this:
return df, df_ground_truth, df_study_type, number_of_patients_entire_database_as_integer, number_of_studies_entire_database_as_integer
then other functions like QUERY_SEMIOLOGY and QUERY_LAT can also return what they already return, in addition to number_of_patients_matching_query, localisation_numbers, lateralisation_numbers, string_of_important_default_exclusions
would that be okay?

mega_analysis or MEGA_ANALYSIS.py?

I meant the mega_analysis package, i.e. the "black box" that Slicer queries (through the semiology API). So yeah, I guess those modifications make sense. You can modify the returns and I'll modify semiology and the Slicer module accordingly. I guess that's the best solution for now.
Please make sure the returned variables and their types are verbosely defined in the function docstring. number_of_studies_entire_database_as_integer and number_of_patients_entire_database_as_integer  could be renamed to e.g. num_studies and num_patients. The meaning and type will be inferred from the docstring.

that was verbose for here to be clear
ok

if the last comment on #69 is implemented, then the only thing remaining for this issue to not need to open slicer log would be to also display somewhere primnently, the total number of patients used to extract localising datapoints.
this is available in semiology module's call to QUERY_SEMIOLOGY(), (within query_semiology)
num_query_loc


[Button] Add Button component and more

Description

Add new Button package
Refactor mineralTheme
Rename style-utils package to component-utils
Update tests suites to snapshot test all demo examples
Misc related updates

Motivation and context
HelloWorld components are nice, but we need to start building real components.  Button is our first real component.
connects #52
How has this been tested?

All examples are now being snapshot tested
Manual visual testing using demo examples

Types of changes

Other (provide details below)

This PR includes a New feature and what would ordinarily be a breaking change (renaming the style-utils package), but since the previous state has not been published, the change is not being specified as "breaking".
Checklist

[x] Renders and functions properly in all supported browsers
[x] Automated tests written and passing
[x] Accessibility and inclusivity considered
[x] Rendering performance (initial load time & 60fps) and perceived performance considered
[x] Documentation created or updated

How does this PR make you feel?

It seems like a lot of these commits save new jest snapshots, I'm wondering how effective they will be in catching regressions.

@limscoder A large portion of the jest snapshot updates here are due to the fact that the mineral component examples are now wrapped with a ThemeProvider for testing.  In the browser, this is provided by the site itself.  While it is quite verbose here and will always be the case when adding new demo examples, smaller changes in the future should not be as large and should help to surface regressions.  That being said, visual-diff testing would also be beneficial.