Enhancement - Performance Optimisation - Import Data

Performance Snapshot

Root Cause

processGraphResponse perform array merge on every single loop execution to combine datasets from the graphList.
combineProcessedData perform removeDuplicates on every single merge against graphFlatten with significant growth rates on according to data size... results in huge consumption in JSHeap.
dispatch is expensive due to the nature of immutability, results in huge execution time on data processing.

Set graphFlatten to [];
For graphData in graphList
 Call dispatch(addQuery) with graphData

 If groupEdge = true
    Call groupEdgeForImportation with graphData and groupEdgeField return groupEdgeGraph and groupEdgeId
        duplicateDictionary
        obtainGroupEdgeIds
        aggregateGroupEdges
        produceGraphWithGroupEdges
        aggregateMetadataFields
        combineEdgeFields
    Set graphData = groupEdgeGraph
    Call dispatch(updateLastGroupIds) with groupEdgeId
 Endif

 Call dispatch(processGraphResponse) with graphFlatten
    combinedProcessData
        removeDuplicates // O(nodes + edges + nodeFields+ edgeFields)
    dispatch(updateAll)
    dispatch(updateSelection)
Endfor

Proposed Solution

Remove duplicates after flatten and merged graph lists.
Derive nodeIds, edgeIds and groupEdgeIds, store into metadata to perform modification on graphFlatten

Set graphFlatten to [];
For graphData in graphList
 If graphData.metadata.groupEdge.toggle = true
    Call groupEdgeForImportation with graphData and groupEdgeField return groupEdgeGraph and groupEdgeDict
        duplicateDictionary
        obtainGroupEdgeIds
        aggregateGroupEdges
        produceGraphWithGroupEdges
        aggregateMetadataFields
        combineEdgeFields
    Set graphData = groupEdgeGraph

    // store the records of aggregation ids for future modification
    Set metadata.groupEdge.fields with groupEdgeDict
 Endif

 // derive node's ids and edge's ids for future modification
 Set nodeIds to GraphData.nodes => node.id;
 Set edgeIds to GraphData.edges => edge.id;
 Set metadata.ids.nodes = nodeIds;
 Set metadata.ids.edges = edgeIds;

 Call mergeGraph(graphData, graphFlatten) return mergedGraph
 Set graphFlatten = mergedGraph
Endfor

// Perform merge with single dispatch
// The Graph List now store all the ids of nodes and edges
Call dispatch(addQuery) with graphList

// Remove Duplicates with single executions after all the graph are merged.
Call dispatch(processGraphResponse) with graphFlatten
    Call combinedProcessData with graphFlatten and existingGraphData return entireGraph
        // O(nodes + edges + nodeFields+ edgeFields)
        removeDuplicates
    Call dispatch(updateAll) with entireGraph
    Call dispatch(updateSelection) with graphFlatten

Metadata's IDs Structure
{
  "nodes": [],
  "edges": [],
  "metadata": {
    "ids": {
      "node": [
        "node-1",
        "node-2"
      ],
      "edge": [
        "group-edge-1",
        "edge-3"
      ]
    },
    "groupEdge": {
      "dict": {
        "group-edge-1": [
          "edge-1",
          "edge-2"
        ]
      }
    }
  }
}

@timlrx pointed out that the proposed solution may removed nodes with same id that exist in both graph list leads to inaccurate graph representation.
After several discussions, we decided to perform the following enhancement and monitor the performance after the refactor:

Reduce the amount of dispatch in single execution due to the nature of immutability.
Refactor combinedProcessData out from the loops because the function possess a minimum time complexity of  4 * O(n+e+nf+ef)^2.


Allow some methods to be omitted from generated swagger docs

I have custom handlers that do not allow some operations on resources and/or relations.  I'd like the generated swagger docs to reflect this.
Here's my proposal.

Interestingly I have certain methods disabled in my adapter disabled, i.e.
var jsonApi = require("jsonapi-server");

module.exports = new jsonApi.MemoryHandler();
module.exports.create = null;
module.exports.delete = null;

Shouldn't the swagger generator also pick those up? As it seems to put them in anyway :(

@adamlc suggestion does appeal and it seems it would be simpler to implement. What do you think @BryceCicada: only add method to the swagger docs if it is defined in the handler?

👍 to @adamlc implementation. I like that it is very clear in the code that those methods are not supported, instead of just forgotten about.

The issue @BryceCicada was trying to solve here is the Swagger documentation of the individual JSON:API "routes" for mutating specific relationships. There are situations whereby we have a resource that can be updated, however it has one or more relationships that aren't modifiable by the API consumer - read only relationships. Right now, if a handler exposes update functionality, the Swagger doc generator will assume all relationships are open to modifications and will generate all the possible documentation.
I've been sat on the fence with this change - I'm not convinced Swagger is the right tool for the job for documenting a JSON:API service, mainly because of issues like this one. I'm also undecided if jsonapi-server should validate attempts to update relationships via the same config as the docs. Basically I'm waiting until this becomes a problem for us in production, hoping that the experiences we're gaining on the journey lead us to an elegant solution.

Makes sense I guess, as regards my comment shall I open another issue? I guess thats a little different to what @BryceCicada wants to achieve.

@adamlc since the scope of your issue seems to be different, it's probably best to open a new issue.

No problem, opened an issue in #190


Truncated labels

If I set the gridlines in my xAxes to false my bar titles are not shown correctly.

My Chart settings:
statisticChart = new Chart(ctx, {
                type: 'bar',
                data: copiedData,
                options: {
                    scales: {
                        yAxes: [{
                                ticks: {
                                    beginAtZero: true,
                                    stepSize:1
                                },
                                gridLines: {

                                }
                            }],
                        xAxes: [{
                                ticks:{

                                },
                                gridLines: {
                                    display:false
                                }
                            }]
                    }
                }
            });

If I set gridLines.display to true. Everything works fine.

Regards Tobi

That's a new one. Should be relatively easy to diagnose.

Great. It' works. Thanks a lot.

What was the solution??


Trivy scanner: Allow pull secrets

Describe the solution you'd like
Some images will not be known to the Trivy scanner. In order to resolve images unknown to the Trivy database, it tries to pull in the images, first using a docker or podman daemon. If that action fails, Trivy will attempt to pull the image by HTTP. In any case, the container will need access to a registry pull secret in the eraser-system namespace.
Environment:
N/A

We should discuss having pull secrets vs running a scanner per node.


Pre-training and fine-tuning scripts

Great work! I find no model pre-training and downstream task fine-tuning scripts in the repository. Could you provide them?
Thanks.

Hi @wuzhen247, thanks for your interest! In the examples folder, we provide a tutorial for training a downstream model to predict the fitness of protein variants.
At this time, we do not provide model pre-training in ESM. Internally, we trained these models using the fairseq toolkit. We highly recommend using fairseq for pre-training new models.
Hope that helps! Feel free to reopen the issue if you have any more questions.

Hi @wuzhen247, thanks for your interest! In the examples folder, we provide a tutorial for training a downstream model to predict the fitness of protein variants.
At this time, we do not provide model pre-training in ESM. Internally, we trained these models using the fairseq toolkit. We highly recommend using fairseq for pre-training new models.
Hope that helps! Feel free to reopen the issue if you have any more questions.

Thanks for your suggestions and fast response. I will try them

Hi;
Thanks so much for your great work; it is so useful
I have a question regarding the above issue:
In the example you mentioned: "Our embeddings are stored with the file name from fasta header: {index}|{mutation_id}|{effect}.pt"
So, how you did that? I mean how you convert the seq in fasta file to emebedded file ( i mean .pt) ?
Thanks again
Nasser

Hi;
Thanks so much for your great work; it is so useful
I have a question regarding the above issue:
In the example you mentioned: "Our embeddings are stored with the file name from fasta header: {index}|{mutation_id}|{effect}.pt"
So, how you did that? I mean how you convert the seq in fasta file to emebedded file ( i mean .pt) ?
Thanks again
Nasser

Hi @nasserhashemi, this is described under "prerequisites" in the example notebook. Pasting below for your convenience.

You have obtained sequence embeddings for ß-lactamase as described in the README, either by:
running python extract.py esm1_t34_670M_UR50S examples/P62593.fasta examples/P62593_reprs/ --repr_layers 34 --include mean OR
for your convenience we precomputed the embeddings and you can download them from here - see below to download this right here from in this notebook

Hi @nasserhashemi, this is described under "prerequisites" in the example notebook. Pasting below for your convenience.

You have obtained sequence embeddings for ß-lactamase as described in the README, either by:
running python extract.py esm1_t34_670M_UR50S examples/P62593.fasta examples/P62593_reprs/ --repr_layers 34 --include mean OR
for your convenience we precomputed the embeddings and you can download them from here - see below to download this right here from in this notebook

Oh, I see, great, Thanks so much for your prompt reply;

Oh, I see, great, Thanks so much for your prompt reply;


add a section on the new melody features from klaus to release notes

Add a section with some notes on the improvements to the sound & thanks.
(@klausw in case I misrepresented any of the changes, the changes were a little while ago so I might have missed some subtleties when reviewing them. And thanks again!)

Thanks! FYI, I just posted https://github.com/microsoft/pxt-arcade/pull/3002 which adds the new waveforms to the documentation. If that gets merged, maybe consider linking to it from the changelog?

Thanks! FYI, I just posted https://github.com/microsoft/pxt-arcade/pull/3002 which adds the new waveforms to the documentation. If that gets merged, maybe consider linking to it from the changelog?

By the way @klausw - my kids and I just spent the last 15 minutes playing around with your new sounds! https://arcade.makecode.com/91960-59425-02296-62940  They're so awesome!!!  THANK YOU :-)

By the way @klausw - my kids and I just spent the last 15 minutes playing around with your new sounds! https://arcade.makecode.com/91960-59425-02296-62940  They're so awesome!!!  THANK YOU :-)

@klausw That PR looks good too! We working through some build issues right now so might have to wait until tomorrow to get both of these docs changes in, but I've updated the attribution ~

@klausw That PR looks good too! We working through some build issues right now so might have to wait until tomorrow to get both of these docs changes in, but I've updated the attribution ~

@klausw That PR looks good too! We working through some build issues right now so might have to wait until tomorrow to get both of these docs changes in, but I've updated the attribution ~

Thank you! Just to avoid misunderstandings, I wasn't asking for that change to get called out specifically, what I meant was that it may be useful to add a link to https://arcade.makecode.com/developer/sound as a pointer for people who want to experiment with the additions.

By the way @klausw - my kids and I just spent the last 15 minutes playing around with your new sounds! https://arcade.makecode.com/91960-59425-02296-62940 They're so awesome!!! THANK YOU :-)

Glad to hear it! I'm currently adding a few more that could be useful for games, I'll update the forum thread once I have something to show.

@klausw That PR looks good too! We working through some build issues right now so might have to wait until tomorrow to get both of these docs changes in, but I've updated the attribution ~

Thank you! Just to avoid misunderstandings, I wasn't asking for that change to get called out specifically, what I meant was that it may be useful to add a link to https://arcade.makecode.com/developer/sound as a pointer for people who want to experiment with the additions.

By the way @klausw - my kids and I just spent the last 15 minutes playing around with your new sounds! https://arcade.makecode.com/91960-59425-02296-62940 They're so awesome!!! THANK YOU :-)

Glad to hear it! I'm currently adding a few more that could be useful for games, I'll update the forum thread once I have something to show.


ES6Promise reference and missing Promise reject handling

Hi there
I have started developing an Angular2 (rc6) app and am using px-login. While working on this, I noticed that my Angular page did not do proper change detection, failing to update the view when a property changed. After a day of puzzling I finally found out that the es6-promise library is interfering with Angular's zones and promise handling. In Angular2, they are now suggesting to use core-js, which already includes a es6 promise shim.
Somehow core-js and es-6 promise bite each other in a way which makes Angular2 fail to update the view silently.
The next step was to remove the es-6 promise reference which I included for px-login to work. This fails since you do a conditional loading on a hard-coded, relative folder name.
//Conditional loading of es6-promise polyfill
    if(typeof ES6Promise === "undefined"){
      var script = document.createElement('script');
      script.async = true;
      script.src = '../es6-promise/promise.min.js';
      document.head.appendChild(script);
    }

In my opinion, this is really bad! You can't just assume that some js file is there in a relative path from inside a Polymer component. Show an error message, log a console message, but don't try and load it magically. It has to be the page / apps responsibility to load a library that implements ES6Promises.
Fundamentally, after commenting this out, my app still works. But I get another error from the getUserInfo function when it tries to get the user info:

Unhandled Promise rejection: Unexpected token < in JSON at position 0 ; Zone:  ; Task: Promise.then ; Value: SyntaxError: Unexpected token < in JSON at position 0

getUserInfo fails and rejects the Promise properly, but the error is not handled in the ready function:
 window.px.auth.getUserInfo().then(function (data) {
                  self.userInfo = data;
                });

This needs a catch like so:
.catch(function(){
    //do something with the error.
})

hi @orendin thanks for the fix to this particular issue of px-login. Sorry for the trouble you had to go through.  We have been aware of this and other items, and have them in our list for px-login.  We are looking into addressing all of them in a possibly major re-work of the component.  Your solution clearly shows best practice, which the next release of the component will adhere to. Please stay tuned for updates.  Thanks again!

I believe this can be closed per the 1.0+ redesign as Henry indicated above.


"Change per (%)" field does not accept decimal values

Hi, I attempted to use this plugin to calculate prices for nature runes from the Yanille rune shop. The appropriate change rate is 0.1%, but the calculator seems to only accepts integers in this field.
Could you please add support for decimal values to this field? Looks like you can just remove this line: https://github.com/JammaBro/Shop-Calculator/blob/a4e496f5573bc60966c70d84fb2852349bb365bb/src/main/java/com/shopcalculator/ui/ShopCalculatorPanel.java#L30C1-L30C6

Changed the input field filtering from integer only to numeric to allow decimal values. Pull request is set for the update on Plugin-Hub so the update should go live in matter of days.


Give users the opportunity to self-delete accounts

Is your feature request related to a problem? Please describe.
According to European privacy laws, users have the right to have their personal data deleted from information systems. Currently, users have to contact the repository administration to initiate removal of accounts and the personal data.
Describe the solution you'd like
A button within the user's profile to allow users to self-delete their account and the attached personal data. This button should work for users that have not submitted their own datasets.
Describe alternatives or workarounds you've considered
As stated above, it is possible for users to request account deletion at the

@arvoConsultores has asked to claim this ticket. Assigned to their team.

What about users that has submitted an item? maybe their finished submissions should be assigned to collection or general administrator?

In my original post, I suggested that the button should only work for users that have not submitted any items. I don't know if that should be up for configuration ´.

I know it was proposed only to unsubmitted users, but before I start makin changes I would like to launch an open question. @tdonohue European privacy laws should be aplied to users with submitted items? if yes how? reassiging it to collection or general administrator? Keeping user but overwriting his personal data with "deleted-user-18"?

@aroman-arvo : My understanding is that in DSpace 7 it is now possible to delete EPersons even if they have submitted content.  See the fixes in #2229 made by @pnbecker .
I cannot answer the question about European privacy laws as I'm not well versed in that.  But, if it does require that people be able to self-delete, then it seems reasonable to allow a logged in person to self-delete their own account.  Since it's already possible to delete EPersons who are referenced elsewhere, then it might be as simple as allowing any EPerson to delete their own account.
It's possible @pnbecker has a better answer here as he's worked in this area of the code much in the past & he is better versed in GDPR and similar than I am.

I think we must be able to delete user. I don't think that users must be able to do this themselves. But I'm not a lawyer and I might be wrong.

I agree with @pnbecker, but I'm also no lawyer. I would be careful with allowing users that have submitted anything to delete themselves. They have, after all, entered into a contract with the repository provider and the provider may want to be able to decide whether they still need information on who this contract is with (according the the GDPR, providers are only required to delete personal information if there is no reason to keep it)., and delete themsleves if the decide they don't.
That's also why I had limited the issue to users that have not submitted datasets.

It is clear then. By default only users without submitted items can auto-delete their user. It is the responsibility of the institution to manage users who have submitted items.
With the work made in #2803 it should be easy.

I want to clarify this ticket a bit further.  I understood @pnbecker 's prior comment as saying "The current behavior may be good enough" because administrators already have the ability to delete users.   Are we certain that Users must be able to delete themselves? Or is it simply enough to document that they can contact their Administrator if they want their account to be permanently deleted?  DSpace already allows users to be deleted, but they simply need to ask their Administrator to make that happen.
I'm also not a lawyer (obviously), but from what I see on GDPR from other sites, the "Right to Erasure" (or right to be forgotten) simply gives users the "right to have your data erased, without undue delay by the data controller".  Quote taken from https://www.dataprotection.ie/en/individuals/know-your-rights/right-erasure-articles-17-19-gdpr
Here's the exact text from GDPR: https://gdpr-info.eu/art-17-gdpr/

The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay where one of the following grounds applies ...

That implies to me that DSpace doesn't need to allow users to delete themselves.  We just need to allow Administrators to be able to delete users, if a user requests it (and we already have that feature in 7.x)
So, I'd like to ask whether we feel the current behavior is "good enough" to align with GDPR.  I'm not a lawyer, but I've yet to find exact text in GDPR which requires us to allow users to delete themselves.

I think the current behavior is good enough.

I think It is sufficient to comply with GDPR.
If there is no interest, for usability reasons, the issue could be closed.

The issue was mainly for usability so that not all users have to contact central support and central support does not have to handle such requests. I agree, to comply with GDPR, the current behavior is good enough.


Changelog of official implementation

Thanks for the well-written package! The RetNet's official implementation had several updates at https://github.com/microsoft/unilm/blob/master/retnet/README.md#changelog .

@donglixp should we use the Microsoft release or this?

@donglixp should we use the Microsoft release or this?

@yhyu13 We didn't conduct a parity check.


Add config attributes to template_fields

I wish to manage kafka_config attribute as template_fields.
I want to have different bootstrap.servers value depending on the deployment environment, such as develop, test, and production. It would be useful while managing different values in the Variable store for each deployment environment with the same DAG code.
For example, in the ProduceToTopicOperator, I can use:
kafka_config={"bootstrap.servers": "{{ var.value.get('kafka__bootstrap_servers')}}"},

Thanks for you contribution - it will go out in the next release.


Improve cli service commands

show all possible information about service
fix service containers list output
update command should not allow all the same options as in create

@nevalla @miskun should we include this in 0.8.0?

Looks good! No reason to exclude this.


Doc updates dist training

Please apply also https://github.com/rh-aiservices-bu/fraud-detection/pull/40 once the changes are merged to dist-training-dev


The compatibility of dexie in IE9 and ie10

Hello, author
As shown in the figure__ proto__ Attribute has compatibility problem in IE9 and ie10. Can you change it to Object.getPrototypeOf (xx)

Is someone using IE9-10? Anyway, Object.getPrototypeOf() is more standard and it's an easy fix. Thanks for reporting!


Find a new paper

Find a new paper "Graph domain adaptation: A generative view" from TKDD-24.
Maybe it can be added on your list.

Ok I will correct the year and journal for 'DGDA'


feat: use build in mappings

Fix for #19
WIP, some mappings need to be ignored.

Closing in favor of https://github.com/pantharshit00/prisma-docs-generator/pull/21


rpiCameraBitrate configuration seems not to take effect

Which version are you using?
v0.22.2
Which operating system are you using?

[ ] Linux amd64 standard
[ ] Linux amd64 Docker
[x] Linux arm64 standard
[ ] Linux arm64 Docker
[x] Linux arm7 standard
[ ] Linux arm7 Docker
[ ] Linux arm6 standard
[ ] Linux arm6 Docker
[ ] Windows amd64 standard
[ ] Windows amd64 Docker (WSL backend)
[ ] macOS amd64 standard
[ ] macOS amd64 Docker
[ ] Other (please describe)

Describe the issue
Description
increase rpiCameraBitrate seems do not take effect
Describe how to replicate the issue
on a rPI4 with picamera v3, configured as 1920x1080 at 30fps.
increase rpiCameraBitrate from 1000000 to 15000000

start the server
publish with rpiCamera
read with webRTC or RTSP

open the stream as webrtc
http://<IP_ADDRESS>:8889/cam
the troughput didn't reach the expected value
same if you try to open RTP stream by VLC
rtsp://<IP_ADDRESS>:8554/cam
I was able to get 3Mbps at maximum
Did you attach the server logs?
no
Did you attach a network dump?
no

answered in #1748


Python Extension hanging

Issue Type: Bug
Downloaded the vs code for mac and drag into the application folder after that went for python extension from microsoft but it keep hanging and not installing.
setting.json file current setting
{
"[python]": {
"/Library/Frameworks/Python.framework/Versions/3.5/bin/python3"
}
}
Thanks
Sharvan
Extension version: 2019.4.12954
VS Code version: Code 1.34.0 (a622c65b2c713c890fcf4fbf07cf34049d5fe758, 2019-05-15T21:59:22.738Z)
OS version: Darwin x64 18.6.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz (4 x 2300)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (10.11GB free)

Process Argv

Screen Reader
no

VM
0%

it keep hanging and not installing.

Please could you be more specific.
Please provide screen shots of the errors

Hi,
here is screen shot when i click install the extension it kept on going as installing but nothing installed.

this what i modified in the setting.json
setting.json file current setting
{
"[python]": {
"/Library/Frameworks/Python.framework/Versions/3.5/bin/python3"
}
}

We do not control installation of extensions in VS Code.
Please report this issue on the VS Code repo.


Make the high-level interface more flexible

we strongly encourage the use of the PeriodicSystems interface

From the docs, it seems that the high-level PeriodicSystems interface is supposed to be used for most purposes.
However, it neither supports non-periodic boxes nor coordinate matrices.
The underlying low-level interface supports both of these, so this is just due to too strict type requirements in the PeriodicSystems interface:

unitcell must be an abstract vector or matrix and can't be a Limits, even though it's supported by Box.
xpositions must be a vector of vectors, even though the CellList supports matrices as well.

@efaulhaber
Sorry for the delay in answering these issues. For some reason I'm not receiving emails about them. I'll look into them all asap.

No worries. Thanks for looking into it!

In version 0.8.23 the PeriodicySystems interface accepts any AbstractVector{<:AbstractVector} as input coordinates, so at least one can reinterpret(reshape, SVector{3,Float64}, m::Matrix), for example, and use matrices as inputs, indirectly.
I will work on generalizing the interface for matrices and check the limitations of the unitcell as well at some point, to make the interface as flexible as the low level one.

On version 0.8.28 matrices are supported as input coordinates in the PeriodicSystems interface. Note that matrices cannot be resized, so the number of particles cannot change when using this.
Missing still the support for non-periodic systems.

Non-periodic systems will be supported in the high-level interface in v0.8.30 by simply not providing a unitcell, or setting unitcell = nothing.
Because of this feature and other code rearrangements, probably I will be settling the interface to be ParticleSystem instead of the now somewhat misnomer PeriodicSystem, for a nearby 1.0 release.
Also the necessity of loading the PeriodicSystems submodule was removed, now using CellListMap provides access to all the functionality. The submodule was kept for now for backward compatibility of the 0.8 series, with a warning.


Add new spacing value

$spacing-layout-72 needs to be added to typography package

Wasn't added as a css variable #38 fixes it

Wasn't added as a css variable #38 fixes it

Fixed in #38

Fixed in #38


Add links to PPP in AboutUs.md

Should be able to print to pdf from these links.

Should be able to print to pdf from these links.

Codecov Report

Merging #284 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #284   +/-   ##
=========================================
  Coverage     64.11%   64.11%
  Complexity      552      552
=========================================
  Files           100      100
  Lines          1831     1831
  Branches        192      192
=========================================
  Hits           1174     1174
  Misses          582      582
  Partials         75       75

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 527df73...0f5a2c9. Read the comment docs.


Integrate Loguru

References: https://github.com/physiopy/phys2bids/issues/461

Integrate Loguru for all log types, including info messages, warnings and exception catching
Proposed Changes

Migrate all message logging to a single logger interface with Loguru. This includes console print()s, warnings and exceptions

Change Type

[ ] bugfix (+0.0.1)
[x] minor (+0.1.0)
[ ] major  (+1.0.0)
[ ] refactoring (no version update)
[ ] test (no version update)
[ ] infrastructure (no version update)
[ ] documentation (no version update)
[ ] other

Checklist before review

[ ] I added everything I wanted to add to this PR.
[ ] [Code or tests only] I wrote/updated the necessary docstrings.
[ ] [Code or tests only] I ran and passed tests locally.
[ ] [Documentation only] I built the docs locally.
[x] My contribution is harmonious with the rest of the code: I'm not introducing repetitions.
[x] My code respects the adopted style, especially linting conventions.
[x] The title of this PR is explanatory on its own, enough to be understood as part of a changelog.
[ ] I added or indicated the right labels.

[ ] I added information regarding the timeline of completion for this PR.

[ ] Please, comment on my PR while it's a draft and give me feedback on the development!

Regarding dependencies, in this PR I just added loguru to requirements.txt. However there were still some packages which were missing from my local environment (e.g. gooey, pandas).
Then I saw #53 and found this proposed line:
To install ``peakdet`` along with the basic required modules just run::

    pip3 install phys2bids

Why not have such dependencies in the requirements file to not require installing phys2bids?

Hey @maestroque! Our main way to install things is through pip (specified and regulated by the setup.py and mainly setup.cfg files). For devs, you can follow this guide:

Currently the logs for errors are very verbose regarding callbacks. Do we want this or should we opt for something shorter? I attach an example below
2024-05-13 13:45:14.880 | ERROR    | peakdet.tests.test_utils:test_check_physio:56 - An error has been caught in function 'test_check_physio', process 'MainProcess' (6353), thread 'MainThread'<PHONE_NUMBER>65312):
Traceback (most recent call last):

  File "/home/maestroque/repos/peakdet/peak-new/bin/pytest", line 8, in <module>
    sys.exit(console_main())
    │   │    └ <function console_main at 0x7f2445c64ae0>
    │   └ <built-in function exit>
    └ <module 'sys' (built-in)>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/config/__init__.py", line 197, in console_main
    code = main()
           └ <function main at 0x7f2445c649a0>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/config/__init__.py", line 174, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
         │     │                │      │    └ <HookCaller 'pytest_cmdline_main'>
         │     │                │      └ <_pytest.config.compat.PathAwareHookProxy object at 0x7f24458d29d0>
         │     │                └ <_pytest.config.Config object at 0x7f24458d2790>
         │     └ <enum 'ExitCode'>
         └ typing.Union
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           │    │         │    │     │    │                  │       └ True
           │    │         │    │     │    │                  └ {'config': <_pytest.config.Config object at 0x7f24458d2790>}
           │    │         │    │     │    └ <member '_hookimpls' of 'HookCaller' objects>
           │    │         │    │     └ <HookCaller 'pytest_cmdline_main'>
           │    │         │    └ <member 'name' of 'HookCaller' objects>
           │    │         └ <HookCaller 'pytest_cmdline_main'>
           │    └ <member '_hookexec' of 'HookCaller' objects>
           └ <HookCaller 'pytest_cmdline_main'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           │    │               │          │        │       └ True
           │    │               │          │        └ {'config': <_pytest.config.Config object at 0x7f24458d2790>}
           │    │               │          └ [<HookImpl plugin_name='main', plugin=<module '_pytest.main' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/sit...
           │    │               └ 'pytest_cmdline_main'
           │    └ <function _multicall at 0x7f244617efc0>
           └ <_pytest.config.PytestPluginManager object at 0x7f2445a2dd50>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          │         │         └ [<_pytest.config.Config object at 0x7f24458d2790>]
          │         └ <member 'function' of 'HookImpl' objects>
          └ <HookImpl plugin_name='main', plugin=<module '_pytest.main' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site...
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/main.py", line 332, in pytest_cmdline_main
    return wrap_session(config, _main)
           │            │       └ <function _main at 0x7f2445d0d800>
           │            └ <_pytest.config.Config object at 0x7f24458d2790>
           └ <function wrap_session at 0x7f2445d0d760>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/main.py", line 285, in wrap_session
    session.exitstatus = doit(config, session) or 0
    │       │            │    │       └ <Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>
    │       │            │    └ <_pytest.config.Config object at 0x7f24458d2790>
    │       │            └ <function _main at 0x7f2445d0d800>
    │       └ <ExitCode.OK: 0>
    └ <Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/main.py", line 339, in _main
    config.hook.pytest_runtestloop(session=session)
    │      │    │                          └ <Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>
    │      │    └ <HookCaller 'pytest_runtestloop'>
    │      └ <_pytest.config.compat.PathAwareHookProxy object at 0x7f24458d29d0>
    └ <_pytest.config.Config object at 0x7f24458d2790>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           │    │         │    │     │    │                  │       └ True
           │    │         │    │     │    │                  └ {'session': <Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>}
           │    │         │    │     │    └ <member '_hookimpls' of 'HookCaller' objects>
           │    │         │    │     └ <HookCaller 'pytest_runtestloop'>
           │    │         │    └ <member 'name' of 'HookCaller' objects>
           │    │         └ <HookCaller 'pytest_runtestloop'>
           │    └ <member '_hookexec' of 'HookCaller' objects>
           └ <HookCaller 'pytest_runtestloop'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           │    │               │          │        │       └ True
           │    │               │          │        └ {'session': <Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>}
           │    │               │          └ [<HookImpl plugin_name='main', plugin=<module '_pytest.main' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/sit...
           │    │               └ 'pytest_runtestloop'
           │    └ <function _multicall at 0x7f244617efc0>
           └ <_pytest.config.PytestPluginManager object at 0x7f2445a2dd50>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          │         │         └ [<Session  exitstatus=<ExitCode.OK: 0> testsfailed=12 testscollected=22>]
          │         └ <member 'function' of 'HookImpl' objects>
          └ <HookImpl plugin_name='main', plugin=<module '_pytest.main' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site...
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/main.py", line 364, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
    │    │                                        │              └ <Function test_new_physio_like>
    │    │                                        └ <Function test_check_physio>
    │    └ <member 'config' of 'Node' objects>
    └ <Function test_check_physio>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           │    │         │    │     │    │                  │       └ True
           │    │         │    │     │    │                  └ {'item': <Function test_check_physio>, 'nextitem': <Function test_new_physio_like>}
           │    │         │    │     │    └ <member '_hookimpls' of 'HookCaller' objects>
           │    │         │    │     └ <HookCaller 'pytest_runtest_protocol'>
           │    │         │    └ <member 'name' of 'HookCaller' objects>
           │    │         └ <HookCaller 'pytest_runtest_protocol'>
           │    └ <member '_hookexec' of 'HookCaller' objects>
           └ <HookCaller 'pytest_runtest_protocol'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           │    │               │          │        │       └ True
           │    │               │          │        └ {'item': <Function test_check_physio>, 'nextitem': <Function test_new_physio_like>}
           │    │               │          └ [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11...
           │    │               └ 'pytest_runtest_protocol'
           │    └ <function _multicall at 0x7f244617efc0>
           └ <_pytest.config.PytestPluginManager object at 0x7f2445a2dd50>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          │         │         └ [<Function test_check_physio>, <Function test_new_physio_like>]
          │         └ <member 'function' of 'HookImpl' objects>
          └ <HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/...
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 115, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
    │               │              └ <Function test_new_physio_like>
    │               └ <Function test_check_physio>
    └ <function runtestprotocol at 0x7f2445d0c9a0>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 134, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
    │       │      │               │             └ True
    │       │      │               └ <Function test_check_physio>
    │       │      └ <function call_and_report at 0x7f2445d0ce00>
    │       └ <method 'append' of 'list' objects>
    └ [<TestReport 'peakdet/tests/test_utils.py::test_check_physio' when='setup' outcome='passed'>]
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 239, in call_and_report
    call = CallInfo.from_call(
           │        └ <classmethod(<function CallInfo.from_call at 0x7f2445d0d120>)>
           └ <class '_pytest.runner.CallInfo'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 340, in from_call
    result: Optional[TResult] = func()
            │        │          └ <function call_and_report.<locals>.<lambda> at 0x7f2412d76b60>
            │        └ +TResult
            └ typing.Optional
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 240, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            │                 │       └ {}
            │                 └ <Function test_check_physio>
            └ <HookCaller 'pytest_runtest_call'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           │    │         │    │     │    │                  │       └ False
           │    │         │    │     │    │                  └ {'item': <Function test_check_physio>}
           │    │         │    │     │    └ <member '_hookimpls' of 'HookCaller' objects>
           │    │         │    │     └ <HookCaller 'pytest_runtest_call'>
           │    │         │    └ <member 'name' of 'HookCaller' objects>
           │    │         └ <HookCaller 'pytest_runtest_call'>
           │    └ <member '_hookexec' of 'HookCaller' objects>
           └ <HookCaller 'pytest_runtest_call'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           │    │               │          │        │       └ False
           │    │               │          │        └ {'item': <Function test_check_physio>}
           │    │               │          └ [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11...
           │    │               └ 'pytest_runtest_call'
           │    └ <function _multicall at 0x7f244617efc0>
           └ <_pytest.config.PytestPluginManager object at 0x7f2445a2dd50>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          │         │         └ [<Function test_check_physio>]
          │         └ <member 'function' of 'HookImpl' objects>
          └ <HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/...
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/runner.py", line 172, in pytest_runtest_call
    item.runtest()
    │    └ <function Function.runtest at 0x7f2445c06520>
    └ <Function test_check_physio>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/python.py", line 1772, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    │    │                                   └ <Function test_check_physio>
    │    └ <property object at 0x7f2445c83880>
    └ <Function test_check_physio>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           │    │         │    │     │    │                  │       └ True
           │    │         │    │     │    │                  └ {'pyfuncitem': <Function test_check_physio>}
           │    │         │    │     │    └ <member '_hookimpls' of 'HookCaller' objects>
           │    │         │    │     └ <HookCaller 'pytest_pyfunc_call'>
           │    │         │    └ <member 'name' of 'HookCaller' objects>
           │    │         └ <HookCaller 'pytest_pyfunc_call'>
           │    └ <member '_hookexec' of 'HookCaller' objects>
           └ <HookCaller 'pytest_pyfunc_call'>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           │    │               │          │        │       └ True
           │    │               │          │        └ {'pyfuncitem': <Function test_check_physio>}
           │    │               │          └ [<HookImpl plugin_name='python', plugin=<module '_pytest.python' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11...
           │    │               └ 'pytest_pyfunc_call'
           │    └ <function _multicall at 0x7f244617efc0>
           └ <_pytest.config.PytestPluginManager object at 0x7f2445a2dd50>
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
          │         │         └ [<Function test_check_physio>]
          │         └ <member 'function' of 'HookImpl' objects>
          └ <HookImpl plugin_name='python', plugin=<module '_pytest.python' from '/home/maestroque/repos/peakdet/peak-new/lib/python3.11/...
  File "/home/maestroque/repos/peakdet/peak-new/lib/python3.11/site-packages/_pytest/python.py", line 195, in pytest_pyfunc_call
    result = testfunction(**testargs)
             │              └ {}
             └ <function test_check_physio at 0x7f2417c61d00>

> File "/home/maestroque/repos/peakdet/peakdet/tests/test_utils.py", line 56, in test_check_physio
    utils.check_physio(fname)
    │     │            └ '/home/maestroque/repos/peakdet/peakdet/tests/data/ECG.csv'
    │     └ <function check_physio at 0x7f241933bba0>
    └ <module 'peakdet.utils' from '/home/maestroque/repos/peakdet/peakdet/utils.py'>

  File "/home/maestroque/repos/peakdet/peakdet/utils.py", line 142, in check_physio
    raise ValueError('Provided data does not have valid sampling rate.')

This seems extremely verbose... Should we give the bare minimum at any level, and then increase the level when a "verbose" CLI call is selected (see phys2bids)?

It can toggled by adding a new logger instance with diagnose=0. Backtraces can be also toggled but they are either on or off, I couldn't find a backtrace level
For the CLI call it can be done using a parameter as you proposed. The example I sent above was also from pytest outputs which have a lot of backtraces anyways.
Another question is if someone just uses the modules on their own as a library instead of the CLI, should we assume that the user then is responsible for their own logger (check here)? Or do we still use loguru?
For the time being I only have it enabled in the CLI and disabled for module use, but the user can explicitly enable it if he wants.
@smoia @m-miedema @me-pic

Another question is if someone just uses the modules on their own as a library instead of the CLI, should we assume that the user then is responsible for their own logger (check here)? Or do we still use loguru? For the time being I only have it enabled in the CLI and disabled for module use, but the user can explicitly enable it if he wants.

That seems sensible to me! Our logging is really meant mainly for the CLI, but consider that it will be a required install once we add it, so our module users should still be able to use it if they want to.

The last issue is that @logger.catch catches all exceptions and logs them in the loguru format. If the logger is disabled those exceptions do not show for the library user even in the standard non-loguru format.
I would propose for @logger.catch to be used as a decorator only for the workflow function of the CLI, to avoid this. Then

CLI users would have everything logged in the loguru format (both logs and exceptions)
Library users would by default not have loguru enabled. If they enable it, only logs will be in loguru format and exceptions will be in the standard format, except if they add a @logger.catch in their own code, as in the snippet below:

with logger.catch():
    peakdet.utils.enable_logger(False, True)
    data = np.random.rand(5000, 2)
    phys = peakdet.Physio(data, 10)

What's your opinion with this? Other than that I believe this PR is ready for review.

Btw a sample log would be something like this with the verbose setting:

and a sample log without verbose

Looks good to me too ! Good job 🎉

Again, seems fair to me. @m-miedema @me-pic what do you think?

Looks good to me!

Here is a look of the logger during a sample "run" of the CLI

That's one with the --verbose option

@m-miedema @me-pic @smoia everything listed in this checklist is completed (+removal of Gooey), and should be ready to merge. Please review

One last change. In some test cases the test was checking if a UserWarning was raised. This expects the warnings module usage and not the loguru logger. The workaround I found was the following diff
---    with pytest.warns(UserWarning):
---        arr = io.load_physio(np.loadtxt(get_test_data_path('ECG.csv')))
+++    arr = io.load_physio(np.loadtxt(get_test_data_path("ECG.csv")))
+++    assert "WARNING" in caplog.text

where also a conftest.py file had to be added, where the caplog fixture where the loguru messages would be redirected is instantiated (check here)
It just tests that a warning message was sent, similarly to the previous version where it just checked for a user defined warning.
The only thing that's different is that if multiple warnings are expected per unit test, this assertion is not enough.
In 86adcf3 I added a "WARNING" occurence count, as deemed more appropriately in each specific test.
assert caplog.text.count("WARNING") == 1

I believe it's a sufficient warning assertion. Let me know your thoughts @smoia @m-miedema @me-pic

One last change. In some test cases the test was checking if a UserWarning was raised. This expects the warnings module usage and not the loguru logger. The workaround I found was the following diff
---    with pytest.warns(UserWarning):
---        arr = io.load_physio(np.loadtxt(get_test_data_path('ECG.csv')))
+++    arr = io.load_physio(np.loadtxt(get_test_data_path("ECG.csv")))
+++    assert "WARNING" in caplog.text

where also a conftest.py file had to be added, where the caplog fixture where the loguru messages would be redirected is instantiated (check here)
It just tests that a warning message was sent, similarly to the previous version where it just checked for a user defined warning.
The only thing that's different is that if multiple warnings are expected per unit test, this assertion is not enough. In 86adcf3 I added a "WARNING" occurence count, as deemed more appropriately in each specific test.
assert caplog.text.count("WARNING") == 1

I believe it's a sufficient warning assertion. Let me know your thoughts @smoia @m-miedema @me-pic (now all unit tests are behaving similarly to the master branch)

This seems like a reasonable solution to me, but @smoia has more experience with how the tests were originally set up here!

While trying to tests the changes using pytest test/ from the peakdet folder, I ran into that error..

ImportError while loading conftest '/home/user/Documents/physio/peakdet/peakdet/tests/conftest.py'.
init.py:19: in
from loguru import logger
E   ImportError: No module named loguru

I've created an venv and installed the requirements, and I am able to see that loguru is there (version 0.7.2)... Do any of you also got that error ?

@m-miedema I added some extra functions to disable the logger and configure the log level. Then also the appropriate warnings and errors are added to point the users to the correct usage of the functions.
Now if you create more than one instances you have a warning, and you are pointed to the correct way to change log level.
Let me know if this works for you!

:rocket: PR was released in 0.4.0 :rocket:


PHP Notice iterating over Google_Service_Webmasters_SitesListResponse::$siteEntry

Since the last vendor update I now see this error/notice running some of my tests:
Symfony\Component\Debug\Exception\ContextErrorException: Notice: Indirect modification of overloaded property Google_Service_Webmasters_SitesListResponse::$siteEntry has no effect
.../vendor/google/apiclient/src/Google/Collection.php:20

The code looks something like this (notice thrown on the foreach):
        $siteListResponse = $googleWebmasterService->sites->listSites();

        $count = 0;
        foreach ($siteListResponse as $item) {
            ++$count;
        }

Environment:
$ php -v
PHP 7.1.7 (cli) (built: Jul  6 2017 13:31:15) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.7, Copyright (c) 1999-2017, by Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans

$ composer show |grep google
google/apiclient                         v2.2.0             Client library for Google APIs
google/apiclient-services                v0.15              Client library for Google APIs
google/auth                              v1.0               Google Auth Library for PHP

Hmm, I am unable to reproduce this. I wrote a test case and it ran just fine
My PHP version:
php -v
PHP 7.1.3 (cli) (built: Mar 28 2017 21:57:28) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies

Thank you for filing this issue. We asked some clarifying questions a week or more ago and never heard back from you. We are unable to proceed with this issue until then, so we are closing it. Please feel free to re-open this issue with more information.


Server-Side transactions

Would server-side transactions be a possibility ? Perhaps even (to not encumber the core) as additional service, or even separate process ?

I was very enthusiastic about OrientDB, up to the point where I dived deep-enough to realize that transactions are only supported client-side (previously I just glazed over "tranactions are there", "oooh, they mention MVCC", and thought "supported"). Nearly everything else is "exactly how I'd wanted it", but for this one giant hole. In some settings you cannot even fully trust the clients. And it is - I imagine - a fuck-ton of finicky work to get it right -- so should it even be offloaded to "driver authors" ?.

Would it be possible to leverage the existing Java Client-side transactions ? Wrap the session in something / stick something in the session, and gain server-side transactions this way ?

Hi,
Transaction in java are both client side and server side, you can do only server side transactions doing a sql script like:
begin;
insert ....
update ...
commit;

like explained here: https://orientechnologies.github.io/docs/3.2.x/sql/SQL-batch.html

I never said I wanted to use Java client-side. Quite the contrary, I'll never write anything new on JVM (the most I'm willing to do is patch mission-critical Java crap).
Even if Java is a popular platform, most of the world will, forever, use something else. IMO if OrientDB is to be considered as serious player on the graph-database playing field, server-side transactions are a MUST for each and every client (must be moved to the server).
I'd re-open this, and assign it some "would-be-nice-in-future" tag.

Shiiiiit, let variable, conditions, waits, loops, my Turing-tarpit heaven-hell, it seems like "if you are willing to learn another scripting language, YOU CAN nearly HAS TRANSACTIONS.
Thanks a lot for the patience,


Artist Id appears incorrect when browsing collection

Hi,
I am trying to write some code that matches new releases with artists in my current collection.
The code is working but there appears to be a mismatch in the Artist.Id from the browse ContenetResponse.
I took Bon Jovi as an example.
When an album appeared in the New Releases endpoint I got the following id:
music.790F0000-0200-11DB-89CA-0019B92A3933
This appears correct and is the same Id shown in the address bar when browsing this artist on the Groove Web Player.
However when I get the content response it shows the following Id:
music.AQILAAAPeQACYm9-5tsBoRw
This is a very different looking Id to any others I have encountered and obviously does not match the one above.
To further add to this the Artist that is downloaded does appear to have the correct Id in its ImageUrl and its Deep Link.
I have added a screen shot showing this. Now I can code around this by getting the Id from the ImageUrl instead but what is the reason for the short Id here and is it a bug?
I have checked multiple artists and they are all showing the wrong (or at least unexpected) Id when using browse.
Cheers

Hi ThePyrobri,
This is expected. The "complex" id you're getting when browsing (music.AQILAAAPeQACYm9-5tsBoRw) internally points to the same content than music.790F0000-0200-11DB-89CA-0019B92A3933, but it has more data encoded in it. This allows us to handle scenarios that mix catalog / collection content. This is indicated by the "Source" property in the response : in the browse call you get the result from your collection, whereas in the new releases call you get the result from our catalog.
I'm thinking of a few ways to make your scenario work. The most stable would be to use the collection id (music.AQILAAAPeQACYm9-5tsBoRw) and make a lookup call specifying that you want the "catalog" source and the response will contain the guid id you expect. But this means making more calls (even though you can batch them). Another solution which doesn't involve making more calls is to either match on Artist Name + Album Name (which is most likely unique) or extract the guid id from the ImageUrl or Link as you suggested.
Cheers,

OK that makes sense. I was thinking that there maybe was some sort of difference between catalog and collection ids but we cant tell from this side of the fence :)
Yes to avoid more calls (I want to process every artist in the collection every time I run a compare) I will just grab the id from the deep Link instead.
Maybe this would be the ideal place to make use the OtherIds property on the main Content object?
I have only ever seen this null so not currently sure if it is in use?

Hi,
We recently added an entry in the "OtherIds" array which contains the "simple" id for each item, just grab it from the "music.catalog" entry:
"OtherIds"  : { "music.catalog" : "music.c61c0000-0200-11db-89ca-0019b92a3933"}
Cheers


Rewrote Memview.Index

Added benchmark tests.
Added a failing test not relevant to our actual usage.
Old implementation:
BenchmarkIndexSmall-8   	   94458	     59803 ns/op
BenchmarkIndexLarge-8   	     337	  17548862 ns/op

New implementation:
BenchmarkIndexSmall-8   	  435463	     12523 ns/op
BenchmarkIndexLarge-8   	    1570	   3629018 ns/op

This could be improved further by searching in parallel for a set of strings; the benchmark roughly reflects how we actually use it.

We are generally in a situation where our buffers are large and our search strings are small.  So it's probably worth using bytes.Index() which has a platform-optimized implementation, and then this checking the len(needle) wraparound cases at each boundary.
That would also solve the correctness problem.
However, it's probably still less efficient than parallel search for the set of strings we're interested in.

New benchmarking:
Previous version in PR:
BenchmarkIndexSmall-8   	 3002158	     11870 ns/op
BenchmarkIndexLarge-8   	    9596	   3624710 ns/op

Most recent:
BenchmarkIndexSmall-8   	12983755	      2619 ns/op
BenchmarkIndexLarge-8   	   23590	   1597876 ns/op

This is by using bytes.Index() instead of rolling our own.
I tried a version of parallel search using regexp but it was much slower (about 10000/30 seconds instead of 23590) so that is not included in the PR.

Re-requested from @thatplguy due to significant changes.


How I can export my data on opensource to migrate or study data

Describe the Bug
Guide is not the same for free opensource full version, maybe incomplete, critical feature: export to import later data How I can export my data on opensource to migrate or study data
Database
PostgreSQL
Relevant log output
No response
Which Umami version are you using? (if relevant)
No response
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response

There is no guide because all the data is in your database. You would just use the database tools. See https://www.prisma.io/dataguide/postgresql/inserting-and-modifying-data/importing-and-exporting-data-in-postgresql


Unused parameter still makes their way to compiled code

The source code:
/**
 * @param {string=} x
 * @constructor
 */
function A (x)
{
	if (x)
	{
		console.log(x);
	}
}

new A();

would be compiled to
new function (a) {
a && console.log(a)
};
As parameter a is not given, it should not be in the compiled code along with all the process using it.

The compiler removes a parameter when it is unused in the function body, not when it is never passed at the call sites. Using info from call sites requires whole-program analysis which is too expensive.

We do call site base parameter optimizations, but it is named based.  If you do:
/**
 * @param {string=} x
 * @constructor
 */
function A (x)
{
    if (x)
    {
        console.log(x);
    }
}

new A();
new A();

You will see that this is produced:
function a(){}new a;new a;

The pass just doesn't handle the case where the function function declaration is inlined into the callsite first (and is thus nameless).


Application Caching architecture

Static files caching

static files are uniquely named during the build process, driven by the content hashes
media (i.e. product images) are also exposed with a unique URL
static files and media can have a far-future-expiration header (in cache-control) as they're unique
backend calls (OCC) can also be cached, but different expiration headers are used per endpoint (i.e. short TTL for price data, mid-term TTL for CMS data)

Edge Caching (CDN)
Both static files, media and backend responses can be cached on a CDN.
Pull strategy is recommended, so that the central cache-control is used to control the  expiry of the edge cache.

Server Side Rendering (SSR)

SSR is used in the first placed for providing a crawling-response for bots, but the response will also be used at the very first visit of the web client (see PWA features in next session how the SSR response is not used in future interactions).
The SSR server wraps the client app in a nodejs (express) server.
As long as the response can be represented based on the URL (and additional cookie info), the SSR response can (and should) be stored on the CDN as well.
The SSR server will use the backend to generate the view. These backend calls wil go through the same CDN.
The generated state in the SSR server will be transferred to the client app in order to avoid rebuilding the state and causing bad experience (flickering).

Cache API (PWA)

As soon as the first (SSR driven) response  is loaded in the web client, the service worker will cache resources based on the service worker configuration.
The service worker will cache application resources as well as content (OCC responses, images).
The next visit will be executed based on the cached resources. In addition, the service worker will verify the accuracy of the index.html in order to see if it's the latest version. If there's a new version available, this version will be used on the next visit.


Add K-Nearest Neighbors

I need to add tests

LGTM! I have only added some minor notes.
Btw, is this a good candidate for a notebook? :)

I think I can try to create a notebook with KNN as a major theme. :)


Not using identifier

This is what I am using
<Disqus shortname="http-website-org" :pageConfig="disqus"></Disqus>

where disqus is comming from data
new Vue({
    router,
    el: '#main',
    data(){
        return {
            // ...
            disqus: {
                url: 'https://website.org/spatial/'+synopsis.id,
                identifier: '/spatial/'+synopsis.id
            }
        };
    },
  // ...
}

But that didn't work. In all pages I see the same comments. So I added the identifier inside the html directly.
<Disqus shortname="http-website-org" :pageConfig="{identifier: '/spatial/'+synopsis.id, url: 'https://website.org/spatial/'+synopsis.id}"></Disqus>

That also didn't work.
I was not using vue-router. But Disqus calls to this.$router. So I added an empty router.
Vue.use(VueRouter);

const routes = [];
const router = new VueRouter({
  routes // short for `routes: routes`
})

Vue.use(VueDisqus, {
    shortname: 'http-website-org'
})

new Vue({
    router,
    el: '#main',
   // ...
}

Hi @neel, thanks for using.
I went up a correction on line 136
Could you test with v4.0.1?

I'm having the same issue. Where the identifier is not being recognized and thus showing all comments. I tested with the most recent version which is 4.0.1.

Could you do an example online and the code of how vue-dials are being used?

Totally will. This is my first time posting to a github project I'm
considering using. Hopefully I can provide value back to your work (re:
hopefully it's not a use error!)
Billy
On Mon, Jul 6, 2020 at 4:25 AM Alan Ktquez<EMAIL_ADDRESS>wrote:

Could you do an example online and the code of how vue-dials are being
used?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ktquez/vue-disqus/issues/42#issuecomment-654175023,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYZDSAMM4BKHCL5VJPHRHLR2GYBHANCNFSM4OPD2QUQ
.

Using<EMAIL_ADDRESS>main.js:
import VueDisqus from 'vue-disqus'

Vue.use(VueDisqus)

Using Disqus in a component:
<template>
  <Disqus shortname="example" :pageConfig="pageConfig"/>
</template>

<script>
   import { Disqus } from 'vue-disqus'

    export default {
        name: 'exampleBlogComments',
        components: { Disqus },
        props: ['content'],
        data: function () {
            return {
                pageConfig: {
                    identifier: 'test',
                }
            }
        }
    }
</script>

No matter what I change the variable identifier, it always loads in the same comment threads. Hopefully somebody can help figure out how I configured this wrong?

So am I same issue, and I ran out another problem is that I want to use pop out window to show the disqus, but every post run out same disqus in per page, and pageConfig didn't work will

@wjfergus
Are you using vue-router?
If yes, are you using history mode?

I'm using 4.0.1, with vue-router and history mode active, and can confirm that it wasn't functioning properly, same kind of error.
I think I didn't setup right at first, and created some comments. It may have bugged something, but when I deleted the comments from the site, setup properly and then tried again, everything worked out just fine.
I also setted the title to be different for every comment page, so maybe there's that too.

I am not using history mode (having difficulty trying to integrate using the vue-cli without having functionality issues. Is this a requirement for this library?

Was able to implement history mode (when in doubt restart your server) and it seemed to solve my issue when implementing title/url/identifier. I was able to implement with just identifier, but am sending title and url anyway.


Moist BC wave + slab + MPI

will address stage 2 and 3 of #39

TODO

[x] automate copying in ClimaAtmos driver + writing wrappers
[x] implement MPI init in coupler
[x] couple slab
clean up code (e.g., "the same Space type but not the same instance" error) https://github.com/CliMA/ClimaCoupler.jl/issues/45 - found a work around, so not needed
[x]   merge CA PRs for BCs https://github.com/CliMA/ClimaAtmos.jl/pull/257
[x]   merge CC PR for spatially varying BC specification
[x]   merge CC PR for TempestRemap online application #CC604
[x]  include the new (ClimaSimulations-excluding) ClimaAtmos interface
[x] conservation checks
[ ] physical checks

The clean-up PR will include

introduce coupler-specific interface abstractions https://github.com/CliMA/ClimaCoupler.jl/issues/44
substitute the latest version of ClimaAtmos interface

@sriharshakandala nice job on ClimaComms.jl implementation! 🚀 I was wondering what's the neatest way of tracking sim time if we want to add performance checks? Do you use @btime?

@sriharshakandala nice job on ClimaComms.jl implementation! 🚀 I was wondering what's the neatest way of tracking sim time if we want to add performance checks? Do you use @btime?

For measuring total simulation times (order of a few seconds or greater), generally speaking, @elapsed should suffice! https://github.com/CliMA/ClimaCore.jl/blob/main/examples/hybrid/driver.jl#L199
@btime is better for microbenchmarking (small pieces of code which take a few ns/micro secs)
@btime is also impractical for measuring full simulations as it runs the function multiple times to gather statistics!

@sriharshakandala nice job on ClimaComms.jl implementation! 🚀 I was wondering what's the neatest way of tracking sim time if we want to add performance checks? Do you use @btime?

For measuring total simulation times (order of a few seconds or greater), generally speaking, @elapsed should suffice! https://github.com/CliMA/ClimaCore.jl/blob/main/examples/hybrid/driver.jl#L199
@btime is better for microbenchmarking (small pieces of code which take a few ns/micro secs) @btime is also impractical for measuring full simulation times as it runs the function multiple times to gather statistics!

Sounds good, thank you!

bors r+


⚠ Secret Website 1 has degraded performance

In e624e6b, Secret Website 1 ($SECRET_WEBSITE_1) experienced degraded performance:

HTTP code: 200
Response time: 3758 ms

Resolved: Secret Website 1 performance has improved in 746a4e2.


1.7.2rc2 - Fresh Sync Wrong Progress Indication

On a fresh sync the progress Number looks strange, and counts down starting from 100000%

After sync is complete everything looks good, its just  during sync.

Thanks for the report - will check if I can reproduce.

I did a setup of a fresh sync and number look OK - so I was not able to reproduce this. I leave this issue open as unconfirmed until the v1.8 release for anybody to report this seeing the same.

Did not got further reports on this. Closing for now.


Preserve single space before self-closing tag

Hello!
I noticed that rule @html-eslint/no-extra-spacing-attrs reports if there a space before self-closing tag.

I think it's more comfortable to be able to customize this behavior.
Maybe it's a personal issue, but the rule is in the "style" category, so guess it should be configurable to fit any preferences.
Maybe this can be covered in separate rule because it's also related to this case in terms of style:

What do you think? Can we implement it? How can I help?

@MorevM  Thanks for the report. :)

Maybe this can be covered in separate rule

I agree. It would be nice if we can handle it.
What about a new general rule no-extra-spacing  which check all extra spaces not only around attributes? Then we can deprecate no-extra-spacing-attrs

@yeonjuan yes, such new rule looks good to me with parameter { spaceBeforeSelfClosing: boolean } or similar.

I'm not familiar with the ESLint API for plugin developers.
While this looks interesting to dig into, I'm a little limited on time so I can't seem to get it done before a week or even two if you don't have time to do it before. :)

@MorevM  Sure, :)  please let me know if you can't take any time to do it. Then I'll take it. 👍

I tried to off this rule by
"@html-eslint/no-extra-spacing-attrs": "off",
but it is still report them as errors.


Change case of 'Tag' component

This PR:

Creates modifiers for .govuk-tag--uppercase and .govuk-tag--lowercase (which also removes the letter-spacing)
Adds .govuk-tag--uppercase modifier to Phase banner component

I think we need to remove the text-decoration / letter-spacing declaration from the component all together, however to avoid waiting for a breaking change I have put in deprecation warnings into the component SCSS and used the modifier in the phase banner even though it's not currently needed.

The lowercase modifier, because it will be used to override lowercase on something that is already uppercase uses :first-letter { text-transform: capitalize; }. This wouldn't be needed if we did the breaking change straight away as we wouldn't need a lowercase modifier.
Let me know what you think is the best approach for this is?

It's maybe a little messy, but the other option might be to add the uppercase transformations to the govuk-phase-banner__content__tag instead – this class already exists, so there would be no markup changes required when users update to retain the uppercase-ness.
Then we would remove them from the tag component, adding a modifier for those that want to continue using uppercase. However:

this wouldn't help for cases where the tag is used for alpha / beta tags outside of the phase banner (e.g. after the product name in the header)
I'm unclear whether making other tags lowercase (excluding the phase banner) would still constitute a breaking change.

(I'm assuming that we want to keep the uppercase transformation for alpha / beta tags – have/should we explored title-casing them too?)

I would vote for something similar to Dave's proposal:

Breaking change
Default is lowercase and no spacing
Uppercase modifier for phase banner usage
Phase banner Nunjucks is updated

Since using the phase banner modifier would work, but as you say there's other places where you'd want the uppercase modifier.


[BUG] CRS parser fails to parse urn:ogc:def:crs:OGC:1.3:CRS84

Describe the bug
I tried to load a geojson with the crs property name of urn:ogc:def:crs:OGC:1.3:CRS84 and it did not work.
To Reproduce
Steps to reproduce the behavior:

create a geojson with the CRS property "crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } }
load geojson as a json
use Geometry.from_geojson to try to create a new Geometry object from the geojson

Expected behavior
The new object should be created successfully.
Environment
Python 3.11.5
Stack trace or screenshots

If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Windows 10

Additional context
I believe the issue is related to this line where the CRS is parsed with a regex pattern: https://github.com/sentinel-hub/sentinelhub-py/blob/c7f59125b802873d28843f089ab177ebbd1d435b/sentinelhub/constants.py#L133
crs_template = re.compile(r"urn:ogc:def:crs:.+::(?P<code>.+)", re.IGNORECASE)
The end of the pattern has two colons at +:: where there should only be one.

HI @garychen2002, thanks for reporting.
It seems like the issue is a bit different, because (according to name schemes) both urn:ogc:def:crs:EPSG::3163 and urn:ogc:def:crs:OGC:1.3:CRS84 are valid CRS definitions. It seems like we should replace :: with :.*: in the pattern.
But in any case, I'll mark this on our to-do list.


[Bug] Can't run spleeter separate because of ModuleNotFoundError

Description
I get a ModuleNotFoundError: No module named numba.decorators when running the separate command.
I have Version 4.8.2 of Anaconda and i have set up the installation in a fresh environment. I have also tried to use a different numba version, since i read somewhere that there is a problem with the decorators in 0.50. I still got the same error.
Step to reproduce

Installed using Conda 4.8.2
Run spleeter separate -i audio.mp3 -p spleeter:2stems -o output
Got ModuleNotFound: No module named 'numba.decorators' error

Output
(py37) D:\Music\Spleeter>spleeter separate -i audio.mp3 -p spleeter:2stems -o output
Traceback (most recent call last):
  File "C:\Users\Rene\anaconda3\envs\py37\Scripts\spleeter-script.py", line 10, in <module>
    sys.exit(entrypoint())
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\spleeter\__main__.py", line 54, in entrypoint
    main(sys.argv)
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\spleeter\__main__.py", line 40, in main
    from .commands.separate import entrypoint
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\spleeter\commands\separate.py", line 15, in <module>
    from ..separator import Separator
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\spleeter\separator.py", line 23, in <module>
    from librosa.core import stft, istft
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\librosa\__init__.py", line 12, in <module>
    from . import core
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\librosa\core\__init__.py", line 109, in <module>
    from .time_frequency import *  # pylint: disable=wildcard-import
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\librosa\core\time_frequency.py", line 10, in <module>
    from ..util.exceptions import ParameterError
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\librosa\util\__init__.py", line 71, in <module>
    from . import decorators
  File "C:\Users\Rene\anaconda3\envs\py37\lib\site-packages\librosa\util\decorators.py", line 9, in <module>
    from numba.decorators import jit as optional_jit
ModuleNotFoundError: No module named 'numba.decorators'

Environment

OS
Windows

Installation type
Conda

RAM available
8GB

Hardware spec
NVidia 1050ti / Intel I5 3570k

I personally encountered the same problem, and I worked around it with conda install numba==0.45.0. Perhaps #347 is related?

@mcpower Oh my goodness, this actually worked. Thank you so much.


feat(t8s-cluster/management-cluster): add config files for k0smotronControlPlaneTemplate

This can be merged when https://github.com/k0sproject/k0smotron/pull/641 is released

The referenced PR was released on friday. I will revieeww


Add Command success message grammatical error

Should be Emergency Contact since the add command only allows user to add 1 emergency contact initlally.

Closed in #144


Add Ayu Dark, Ayu Mirage, and Ayu Light theme

I have ported Ayu theme to Lite XL. Can I list it by pointing to my repo?

Can I list it by pointing to my repo?

We'll have to add a manifest.json to this repo to make it work with https://github.com/lite-xl/lite-xl-plugin-manager then.
I'll add one later, and will reply back with instructions on how to add your repo to it.

Added in the remote entries for this!


Add preflight controller implementation (#158)

This PR implements the preflight controller logic
and filtering

add filter function for getting preflights
on modules change
add controller implementation
unit-test

/cc @ybettan

Upstream PR for cherry-pick:
https://github.com/kubernetes-sigs/kernel-module-management/commit/8f828a76a1f0ec75b215c1da82d27ef053ab0f95

/lgtm


INSERT IGNORE INTO is better than INSERT INTO

Use the INSERT IGNORE command rather than the INSERT command. If a record doesn't duplicate an existing record, then MySQL inserts it as usual. If the record is a duplicate, then the IGNORE keyword tells MySQL to discard it silently without generating an error.

Hi,
You can use query method for specific queries.


Changed password, network no longer reachable

I changed the password of my network. Now I get errors in the home assistant integration which still persists even when I change the password back to what it was.
2023-11-10 23:16:17.940 INFO (MainThread) [CasambiBt._casambi] Trying to connect to casambi network 41:...:F1...
2023-11-10 23:16:17.941 INFO (MainThread) [CasambiBt._keystore] Loading keys...
2023-11-10 23:16:17.943 INFO (MainThread) [CasambiBt._keystore] Loaded 1 keys.
2023-11-10 23:16:17.943 INFO (MainThread) [CasambiBt._network] Loading session...
2023-11-10 23:16:17.944 INFO (MainThread) [CasambiBt._network] Loading unit type cache...
2023-11-10 23:16:17.945 INFO (MainThread) [CasambiBt._network] Getting network id...
2023-11-10 23:16:18.159 INFO (MainThread) [CasambiBt._network] Got network id 64i...xD.
2023-11-10 23:16:18.159 INFO (MainThread) [CasambiBt._network] Updating network...
2023-11-10 23:16:18.162 INFO (MainThread) [CasambiBt._network] Loaded cached network. Revision: 24
2023-11-10 23:16:18.202 ERROR (MainThread) [CasambiBt._network] Update failed: 410
2023-11-10 23:16:18.202 ERROR (MainThread) [custom_components.casambi_bt.config_flow] Unexpected exception
File "/config/custom_components/casambi_bt/config_flow.py", line 139, in async_step_user
File "/config/custom_components/casambi_bt/config_flow.py", line 55, in _validate_input
File "/usr/local/lib/python3.11/site-packages/CasambiBt/_casambi.py", line 138, in connect
File "/usr/local/lib/python3.11/site-packages/CasambiBt/_network.py", line 197, in update
CasambiBt.errors.NetworkUpdateError: Could not update network!

Any idea how to fix it?
To be honest, the password flow in the casaba app is not that reliable. I changed the password, but it looks like it's not able to persist it, the field is empty after I store it.

I think you invalidated your session or something and this isn't getting picked up. Is it fixable by deleting the cache?

The cache in my local browser, or somewhere in HA?
Would be nice if every error bubbles up to HA as hand full of exceptions.

The casambi-bt-store folder that should be placed in your HA config folder.
And I agree that exception handling is quite messy currently.

Nice, deleting the network folder solved the issue. This should be done when integration is removed, no?

I would say it should be done when the integration is configured. But there are two bugs here. Cache invalidation should be implemented but the library should also recognize that the session it is using is expired.

The location is determined in the library, right? I think it would be better if the ha integration would store its data in .storage/casambi-bt.` or something like that as it's done for some other integrations. That way we're not polluting the directory visible to the user.

Yes the library just uses the CWD. I don't think hardcoding a dependency to a HA path is a good idea so the path would have to be customizeable

Yes, defaulting to cwd is fine, but would be nice if this could be changed in HA.

Thank you for this! I was encountering the same issue, and deleting the casambi-bt-store directory in config was the thing that fixed it.


[HUDI-5380] Fixing change table path but table location in metastore …

If we change the table path, the table location in metastore will not change after hive-sync. This PR fixes the behavior.
Fixes #7422
Change Logs
Add some code to change the table's storage descriptor when running hive-sync.
Impact
none
Risk level (write none, low medium or high below)
none
Documentation Update
none
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

I will add uts later.

@codope I will be appreciative if you help me review.

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

The failure of CI seems to have flaky test.

@danny0405 can you do reivew for me? I wll be appreciate.


[Feature Request] Ability to restart streaming after stopped

Hello,

I using WebRTC with FullyKiosk browser on tablet.
Network connection to HA server using wired ethernet.
Sometime WebRTC stop streaming and need to refresh Lovelace to restart WebRTC although no connection drop from wired.
Is it possible to add ability to restart streaming after it stopped or add timeout to reset connection after error occurred ?

It should use auto restart already

First time when I create this topic.
My camera entities configure using UDP transport and I though this can cause interrupt when streaming.
Now I changed to TCP, Streaming seem smoother but it just suddenly stop on 17:00
https://www.youtube.com/watch?v=eOaaFjEG0hs
I record video to show you It not restart automatically. It just spinning indicator forever.
As you saw uptime. It work well for a day without stop.
And there are no network & camera connectivity issue from HA on 17:00

Update: Now It's 19:06 and still not reset.

You can check debug logs for more info
https://github.com/AlexxIT/WebRTC#debug

Hello,
Just not working today on 19:30.

Debug log already turn on but no relate error about WebRTC in that time.

Button "Load full log" under this window

Sorry I didn't notice that.
Here full logs.

This is error from main Home Assistant code. I don't know if it because of WebRTC. But seems like it breaks WebRTC component.

The lasted error from supervisor It came from when I update supervisor today.
Not sure about EOF message. Is that relate to keyframe ?
Since my substream using default 50 with framerate 20.

I will try to restart HA to clear log and collect info again tomorrow.

websocket EOF it is normal situation. Lovelace breakes connection to server when WebRTC starts streaming.

I see, Can you tell me which line that Lovelace breaks connection. So I will ask the other next.

Today it stopped on 22:09.

This is HA log (I didn't do anything with HA this day)

Seem like It always stopped after 1 day of streaming.

I am experiencing this problem too. Every once in a while the image gets lost on my dashboard. Refreshing the screen reloads the camera correctly.

Sorry to hear this. I'm give up and roll back to picture entity instead.
Since the debug log can't find anything relate to this and I cannot fix this by myself.
Actually I just need ability to auto refresh playback after message Error: No Video appear.

I have the same issue, maybe for a short term workaround we can use a browser plugin to automatically refresh the screen every N minutes.

Should be fixed in v3

Pretty sure I'm still experiencing this: lovelace in Fully loads fine, then after some time the video stops and requires a refresh of the entire page to start again. Is there something I can do to debug/fix?


Set the minimal TYPO3 12LTS version to 12.2

TYPO3 12.2 has been released. So the 12.0 and 12.1 sprint releases are no longer relevant.

@sascha-egerer ping

Sorry, forgot to take care


Portfolio works section done

Milestone - Works section completed
In this pull request I implement the following features:

Added works cards with his respective information
Added about me section
Added my resume section where I put Languajes, Frameworks and Skills that I been working with
Added a simple form

Thank you for your feedback I'm doing all the changes that you suggest!

good job @NicolasAndrehh

Hey @Tadesse-Alemayehu!
I already do all the changes that you suggested, do you think it's well done?


file is not a database

hello, I have a problem with MonkeyCache.SqlLite 1.2.0 and I use an android api 26
I try to use this code
Barrel.ApplicationId = "Cache";
Barrel.Current.EmptyExpired ();
string cache = Barrel.Current.Get  ("Visits-" + id_bn);
and an Exception is thrown: "file is not a database"
Could you help me?

I am unable to reproduce this at all here.


Hot reloading fails when wrapping a component

Let's say you're going from
export default () => <div />

to
import wrap from './some-decorator'
export default wrap(() => <div />)

The symptom is that all styles and head items are lost. Probably related to the identity of the component?

I can't reproduce. maybe https://github.com/zeit/next.js/pull/680 fixed the issue ?

Likely


show both table and column information under the asset

Is your feature request related to a problem? Please describe.
I did check 1.1. For example term First Name (https://sandbox.open-metadata.org/glossary/Banking.Account) has been tagged to table https://sandbox.open-metadata.org/table/sample_data.ecommerce_db.shopify.raw_customer columns comments and creditcard.
Yes, I can click on the preview, but because the columns in the question have a few tags - I don't see the tagged term just a  +3 more and +4 more https://sandbox.open-metadata.org/glossary/Banking.Account/assets:

Describe the solution you'd like
Ideally, under the asset tab, it would display the column info. Linking the business term to a column (is this what was tagged to) is important for UX. because this is what I am looking for.
this would be very useful to see that the same business term is being called differently across the business applications. for example: customer_id, client_number, consumer_id, partner_number.

Describe alternatives you've considered
using both tags (for linking the asset) and adding technical name as synonym (for user information).
Additional context
Add any other context or screenshots about the feature request here.

@karanh37 can you take this up? as part of our ongoing asset page changes


Automatic marking for custom buildings and bridges

initServer functionality to automatically place markers representing custom buildings and bridges for consistency

 Auto mark custom buildings script

Should be reviewing and commiting this tonight


Error using own Model with non-quadratic images

Hey there,
I use the following model:
def model(train_generator, validation_generator):

    training_steps = training_samples // batch_size
    validation_steps = validation_samples // batch_size

    input_shape = (img_width, img_height, 3)

    model = Sequential()
    model.add(Conv2D(16, (3, 3), input_shape=input_shape))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Conv2D(32, (3, 3)))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Conv2D(64, (3, 3)))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Flatten())
    model.add(Dense(64))
    model.add(Dense(64))
    #model.add(Activation('relu'))
    #model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense(nb_classes, activation='sigmoid'))
    #model.add(Activation('sigmoid'))
    model.compile(loss=['categorical_crossentropy'],
                  optimizer='adam',
                  metrics=['accuracy'])

    model.fit_generator(
        train_generator,
        steps_per_epoch=training_steps,
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps=validation_steps)

    score, acc = model.evaluate_generator(validation_generator, steps=validation_steps)
    #...
    return model

My img_width and img_height are defined as img_width, img_height = 320, 180. If I try to use the example code, I get an error.
new_model = to_heatmap(model, input_shape=(320, 180, 3))
Error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-daa853f5145e> in <module>()
     31
     32 model = model.model
---> 33 new_model = to_heatmap(model, input_shape=(320, 180, 3))
     34 display_heatmap(new_model, "data/test.jpg", 2)

heatmap\heatmap.py in to_heatmap(model, input_shape)
    254         # If not the last layer of the model.
    255         if index + 2 != len(model.layers) - 1:
--> 256             x = add_reshaped_layer(model.layers[index + 2], x, size, atrous_rate=atrous_rate)
    257         else:
    258             x = add_reshaped_layer(model.layers[index + 2], x, size, atrous_rate=atrous_rate,

heatmap\heatmap.py in add_reshaped_layer(layer, x, size, no_activation, atrous_rate)
    210     x = new_layer(x)
    211     # We transfer the weights:
--> 212     insert_weights(layer, new_layer)
    213     return x
    214

heatmap\heatmap.py in insert_weights(layer, new_layer)
    155     W, b = layer.get_weights()
    156     ax1, ax2, previous_filter, n_filter = new_layer.get_weights()[0].shape
--> 157     new_W = W.reshape((ax1, ax2, previous_filter, n_filter))
    158     new_W = new_W.transpose((0, 1, 2, 3))
    159

ValueError: cannot reshape array of size 3112960 into shape (20,20,64,64)

I think, the reshape is somewhat wrong. If I look into the summary of the model I get:
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1_input (InputLayer)  (None, 320, 180, 3)       0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 318, 178, 16)      448
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 159, 89, 16)       0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 157, 87, 32)       4640
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 78, 43, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 76, 41, 64)        18496
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 38, 20, 64)        0
_________________________________________________________________
flatten_1 (Flatten)          (None, 48640)             0
_________________________________________________________________
dense_1 (Dense)              (None, 64)                3113024
_________________________________________________________________
dense_2 (Dense)              (None, 64)                4160
_________________________________________________________________
dense_3 (Dense)              (None, 10)                650
=================================================================
Total params: 3,141,418
Trainable params: 3,141,418
Non-trainable params: 0
_________________________________________________________________

If I calculate 20 x 20 x 64 x 64 this is not 3112960. However, if I look into the network summary, one can see that the size is in fact 38 x 20 x 64 x 64 and this is 3112960. I had a short look into the source code, but it seems to respect the different width and height:
ax1, ax2, previous_filter, n_filter = new_layer.get_weights()[0].shape
new_W = W.reshape((ax1, ax2, previous_filter, n_filter))

If I try to use a quadratic size, i.e.  224 x 224, it works flawlessly.
Am I overlooking something or is this a bug?
Best
hija

It seems that you were using a sequential model and my script didn't handle sequential models before. I've updated it. Can you give a try to the updated version and tell me if it worked for you? Thanks!

Hi there,
unfortunately this does not resolve my issue. I tried with the same code from above and I get the same error. 😞


The click action for returning to the lobby should appear more often

Currently, it only shows at the very start of the game, when you complete a goal and since beta2 when you rejoin the game.
It should additionally show up when an opponent team completes a goal, and when you die.
Possibly it would also make sense to send the message on a timer periodically in order to further minimize players losing access to it or getting confused on how to return to the lobby.

One idea is perhaps to prefix the tellraw messages in chat with an exit symbol that has a hover/click event to return to the lobby.
Something like an arrow pointing to a door maybe?
e.g.

Pros:

shows up more often without adding additional spam to the chat.

Cons:

might not be often enough if there are a lot of users typing in chat?
might not be immediately obvious to the user, unless it is explained to them beforehand.

Item gets are quite frequent, so if many people are chatting, you should also expect more item gets, don't think that would be an issue.
I just think it's not obvious enough and it's not really in a place where you'd expect it.
It becomes even more convoluted when more actions get added such as entering spectator mode (#70).

I like that idea.
I think I would still prefer to send the full message in some situations (Rejoining I don't see any other way since there is no message send then anyways), but I also think I'd prefer it after completing a goal since it's very common to use these options then.
The 3 bars should be added to the GO! message on game start and every item get message. If you have other places in mind as well, there might be other things I am not thinking about.
Also, not sure when the Hint: text should appear now. Currently it is intentionally only on game start to reduce chat clutter, but if it's player triggered, I guess it could just always be when you press the 3 bars (but not when the full message is send by default)

Beside deciding where to show current hint-lobby,
it'd be also good to add hint-menu (about using ≡) that displays for sure for the rejoining players, game start and maybe in other moments


Add pkgcc.sh script

Signed-off-by: James Taylor<EMAIL_ADDRESS>

Please always add a PR/commit message explaining the change.

Hi @denyeart, I just opened this draft for discussion with Josh. I'll update the commit if it ends up going anywhere.

Closing this in preference for the tactical fix to peer lifecycle chaincode package discussed in #3368
Note: there is currently a similar pkgk8scc.sh script available