Docmentation EVC04

Describe the bug
It should be mentioned in the documention, that you need a model from the "Home Smart" or "Connect Plus"-Series.
The EVC04 Homeplus serie can not be controlled, because neither ethernet nor modbus is available
see https://www.vestel-echarger.com/EVC04.html
Steps to reproduce
None
Configuration details
None

Log details
None

What type of operating system are you running?
Linux
Version
No response

Closed in https://github.com/evcc-io/evcc/commit/af75c52db1a733953adfb9db71a508aea2250a56


/usr/bin/env: ‘node\r’ no such file or directory

Expected behavior and actual behavior.
After installation and trying to run nem2-cli I get this error:
/usr/bin/env: ‘node\r’ no such file or directory
Google says it has something to do with the wrong encoding of the endings in the nem2-cli file(s).

Steps to reproduce the problem.
Install nem2-cli and run nem2-cli

Specifications like the version of the project, operating system, or hardware.
Using latest version of nem2-cli and node v13.8.0

Thanks for reporting. At this moment, the only node version supported is Node.js 12 LTS https://github.com/nemtech/nem2-cli#requirements
I leave this issue open to keep it in mind when adding support for Node.js 13.

@meyns This issue should be solved now that we are deploying from Travis. If you experiment the same problem, do not hesitate to open the issue again.


How to use docker image for kics pre-commit hooks?

Is your feature request related to a problem? Please describe.
The v1.5.2 release deprecated binaries and recommended moving to docker. That's great, I'd love to. I'm on a Mac and I already use docker daily.
So I went to the docs site to look for the updated instructions, and made sure I'm on the v1.5.4 version of the docs:
https://docs.kics.io/latest/integrations_pre_commit/
So I'm looking and... It doesn't say anything about using docker in this case. Am I expected to make script in my path called kics in order to run docker run --rm -v /path/to/some/dir:/tmp -t kics:latest or whatever?
Describe the solution you'd like
I'd really like to upgrade because v1.5.1 seems to have a bug I can't currently get around and we have kics in our pre-commit hooks. But the pre-commit hooks at v1.5.4 seem to still use my binary from v1.5.1 since I can't upgrade it on homebrew. And there's no documentation covering this presently, so maybe if you could please add that first, and then help me if I have troubles once the docs exist, that would be wonderful.
Describe alternatives you've considered
Tried to figure out how to make pre-commit use docker, but failed.
Additional context
I can't wait to try this out, so please let me know if I can help somehow. I don't mind writing the docs if you have some instructions somewhere.

Hello, @tspearconquest, I think using docker_image language can solve your issue while docs are not updated. 🙂

Hi @tspearconquest, thank you for contacting us!
There is a PR to fix the docs and to create the kics pre-commit hook.
Meanwhile, you can use docker_image language to create your own local hook, as @lipeavelar said. Here is an example:
repos:
    - repo: local
      hooks:
          - id: kics-scan-local
            name: Kics scan
            language: docker_image
            entry: checkmarx/kics scan -p /src --no-progress
            verbose: true

Actually, by reading pre-commit docs, they already set the image workdir to /src, so you can run as you said and even without the -w flag.
Example:
repos:
    - repo: local
      hooks:
          - id: kics-scan-local
            name: Kics scan
            language: docker_image
            entry: checkmarx/kics scan -p . --no-progress
            verbose: true

I assume you meant: https://pre-commit.com/#docker which is not docker_image language but rather it is docker language.

I tried it, it works. Thanks!


Bootstrap buttons compatibility

Don't have time to explain, watch :)
http://jsbin.com/hefayoxoba/1/edit?html,output

Fixed in 1.22


Fixed unsafe ERC20 transfer at SureToDo::_transferERC20To

some ERC-20 tokens does not stipulate that transfer must throw if the message sender holds insufficient balance. Instead, returning false is compliant with ERC-20 and implemented by many tokens, including BAT, cUSDC, EURS, HuobiToken, ZRX and many more.
additionally some tokens like USDT on ethereum mainnet do not return a boolean value if transfer was successful or not.

This is addressed in PR #3.


styles.min.css not found using cross links

I'm fairly new to TW so sorry if this is my error. When I use the environment variable method of installing cross links globally in a Node.js installation I get the error that styles.min.css is not found. Copying styles.css to that name fixes it.

This project uses Tailwind CSS. The CSS files under src/ are incomplete; we generate the final CSS in a build step.
Copying styles.css to styles.min.css will fix that error message, but it won't actually install the intended style sheet. If you drag the plugin to your own wiki instead, you'll get the generated CSS.
I hadn't considered the plugin directory installation method because I don't use the Node.js version. Do you need to use the directory method for a particular reason?

Thank you for your explanation. I'm trying to wrap my head around the Node.js version but it seems like there is no real downside to just importing by dragging into the wiki.


Production instance to do

-Ensure nightly copies of data directory for database to Azure's storage server.
-Change database password.
-Test deployment process to ensure database isn't wiped on asrg-app/asrg-nginx deploys.
-Set debug to false.
-Setup DNS/install certs
-Populate database (i.e. past events, existing job opportunities, announcements, and admin users)

Bogdan can create scripts to import the data from CSV files.  John should provide CSV files.


New workbench testing from code 4/26/2022

I have built and been testing the new workbench, starting with very simple views.
So far I have found a couple of issues:

I don't seem to be able to create new environments under administrator/new (database error)
I don't seem to be able to do "save as" of an existing demo view which I had successfully imported first
some null pointer exceptions in seemingly random places maybe resulting from attempting 1) and 2)

I am able to do "save as" for physical files, logical files and logical records. I am able to create a simple new view from scratch which I exported and copied to z/OS. This ran fine.

Some other strange things:

I can delete a view which I created from scratch, but I can't delete a look-up path which I created as a save as. Is there a difference between these ?
Having deleted the view I created from scratch I can't get it back from the exported XML, because I can't create separate environment to put it in.

Thanks for the code updates Ian.
I had build problems I don't understand after doing a successful git pull. But created another local branch in which to continue.
So I am now able to perform "save as" on views and look-up paths successfully.
However, I am not able to perform admin/new/environment -- same error:
Caused by: org.postgresql.util.PSQLException: ERROR: query is not a SELECT
Where: query: INSERT INTO gendev.environ(
environid, name, comments, createdtimestamp, createduserid, lastmodtimestamp, lastmoduserid)
VALUES ( DEFAULT, P_ENVTNAME,P_ENVCOMMENTS,
CURRENT_TIMESTAMP,P_USERID,
CURRENT_TIMESTAMP,P_USERID)
RETURNING environid

Now able to create new environments thanks to Ian !!
Still trying to figure out why successfully exported and tested (on SP13) view would not re-import.

Ian, in my version of workbench -- with changes you made available and having created another environment -- I am still getting "fail" when I:

import the DemoOpen view folder and make the views active
modify and "save as" one of the views [10689] and then make it active
export this as an xml document successfully and I can see it
attempt to reimport it to another environment.

Git would not let me attach an xml file so I sent it to you in slack..

Exporting active view created as "save as" from environment "Development" and trying to import it into new environment "new26" results in failed import (result==fail).

This should be sorted following PR #63

This now works: by doing a fresh "save as", exporting the view and then importing it into a different environment. Build as of 5/10/2022.

@neilbeesley Could you comment if these are still issues?


how to store the results of Community.cluster

Currently, the Community class has a method cluster which conducts neighborhood segmentation (clustering) on selected census variables.
The results of the neighborhood cluster labels is appended to a copy of the  original geopandas dataframe as an additional column. Then, a new instance of Community is created using this dataframe and returned to the user.
Two issues with this strategy:

users won't be able to know which variables are used to construct the neighborhood segmentation since we are returning the full original data table.
some tracts (rows) are abandoned because of the missing values in the selected variables and years, and this is hidden from users if we return the full data table.

this is a metadata problem, as i understand it, right? When a user creates a custer analysis from a community, they get back a new community with cluster labels attached, and the question is whether we need additional metadata handling?

users won't be able to know which variables are used to construct the neighborhood segmentation since we are returning the full original data table.

is this kinda like using the dataframe columns as implicit metadata? so like, the user will know which variables were part of the cluster analysis because they are the only variables left on the dataframe of the new community? That would solve the problem but there are a few things i dont like about that approach.
First, i dont like removing valuable data as a way of managing metadata. Sometimes you want to use the cluster labels to summarize other data you have for a community without including it in the cluster analysis (e.g. clustering on race and income, but then examining the median age in each of the resulting clusters to see if other patterns emerge). You could attach it with a join, but i'd argue that's a bigger hassle than just giving the user back what they put in. I'd rather remember which columns I used for the cluster than have to merge more variables back (but i could be in the minority...).
Second, when you call cluster it returns a new community, so an alternative way to handle this would be to just store communities with different cluster params in separate variables. Since you can return the cluster instance itself (which includes all the metadata about which variables were used, which k, etc) along with the new community, you could use that option to build a dict or something of like {community: clusterer} with different params with each associated set of cluster labels, which might be different way to solve this problem?
Third, it seems conceptually very similar to the second problem that you raise, which is that once you've defined a community, it's reasonable to keep all the data about it intact, including which geoms are within its extent and which variables you have to describe it. I dont like the idea of dropping variables any more than i like the idea of dropping observations.
to that end, I agree this is not ideal:

some tracts (rows) are abandoned because of the missing values in the selected variables and years, and this is hidden from users if we return the full data table.

ideally we should return all the tracts and just have NAs for cluster labels if the input data contain NAs... I could swear this is how it's currently implemented but i need to double check. I remember there being some problems with the spatial clusters.
so i think this is kinda a question of philosophy. Do we want to maintain the integrity of the community and keep all cases and variables? Or do we want to give primacy to the cluster analysis and remove variables and cases that dont apply to that particular cluster instance? I definitely vote for the former, understand arguments for the latter, and would welcome more input. I think we just need to keep consistency (i.e. keep rows and columns or drop both).

so i think this is kinda a question of philosophy. Do we want to maintain the integrity of the community and keep all cases and variables? Or do we want to give primacy to the cluster analysis and remove variables and cases that dont apply to that particular cluster instance? I definitely vote for the former, understand arguments for the latter, and would welcome more input. I think we just need to keep consistency (i.e. keep rows and columns or drop both).
I would vote for the second strategy. It is important that we reveal the computation process to the users:

what observations are dropped/kept because of NAs
what columns (variables) (and also rows if not all available census years are selected) are selected and thus responsible for the clustering output

Users already have the original full data - it appears redundant to me that we return the original table to them.
Either way, helper functions for further data transformation will be useful: for long-to-wide transformation, wide-to-long transformation, merge, etc. These functions/methods should be specific to the Community class and will have more meaningful names than the pandas "counterparts".

what observations are dropped/kept because of NAs

if we adopt the first strategy, you know which variables were dropped because they dont have a cluster label

what columns (variables) (and also rows if not all available census years are selected) are selected and thus responsible for the clustering output

the user knows which columns are part of the cluster analysis because they just chose them in the last call to cluster. It's the user's responsibility to know to know their dataset and which variables are available in each year

Users already have the original full data - it appears redundant to me that we return the original table to them.

it's very likely you want to do analysis with other variables beyond those used in the cluster analysis, so it's not redundant to include them it's a hassle to remove them because then the user has to go through the trouble of adding them back. Otherwise, why wouldnt you just use scikit-learn or another cluster provider directly?

Either way, helper functions for further data transformation will be useful: for long-to-wide transformation, wide-to-long transformation, merge, etc. These functions/methods should be specific to the Community class and will have more meaningful names than the pandas "counterparts".

definitely agree, though we'll need to think this through carefully, since a Community is a long-form dataset by definition

resolved by #158


n_wins

hi, i have been going through the code and paper to get a thorough understanding of the internal mechanisms of the architecture. one thing that has eluded me is n_wins in the forward pass of many modules.
can anyone give an insight as to what this n_wins is and what an expected input might be so i can simulate a forward pass with a random tensor?
thank you

n_wins is the number of spectrogram segments/windows, which depends on the duration of the input speech sample and the hop size. It comes from this function: https://github.com/gabrielmittag/NISQA/blob/47d8fead6ab62f62baf0b740a7d1781590a93c0d/nisqa/NISQA_lib.py#L2239
Basically, it is needed to make the model work with speech signals of different durations. You could also download the speech corpus and run the example scripts to get a better understanding of it.

thank you Gabriel


support TF2

solve issue #8

support TF2's keras model

detect model's channels ordering and convert ordering automatically

hi,
thanks a lot for the contribution!
I'll try to quickly test and merge it.

It looks good to me. Would it be possible for you to add an example (including any small model with checkpoints) of how to use it (as in https://github.com/fra31/auto-attack/tree/master/examples)?

It looks good to me. Would it be possible for you to add an example (including any small model with checkpoints) of how to use it (as in https://github.com/fra31/auto-attack/tree/master/examples)?

Sure, I will give TF verion's examples.

@fra31 I have finished TF version's example
There TF1 version's output:
initial accuracy: 97.74%
apgd-ce - 1/10 - 98 out of 1000 successfully perturbed
apgd-ce - 2/10 - 129 out of 1000 successfully perturbed
apgd-ce - 3/10 - 102 out of 1000 successfully perturbed
apgd-ce - 4/10 - 102 out of 1000 successfully perturbed
apgd-ce - 5/10 - 98 out of 1000 successfully perturbed
apgd-ce - 6/10 - 74 out of 1000 successfully perturbed
apgd-ce - 7/10 - 54 out of 1000 successfully perturbed
apgd-ce - 8/10 - 55 out of 1000 successfully perturbed
apgd-ce - 9/10 - 28 out of 1000 successfully perturbed
apgd-ce - 10/10 - 74 out of 774 successfully perturbed
robust accuracy after APGD-CE: 89.60% (total time 47.2 s)
apgd-dlr - 1/9 - 10 out of 1000 successfully perturbed
apgd-dlr - 2/9 - 20 out of 1000 successfully perturbed
apgd-dlr - 3/9 - 15 out of 1000 successfully perturbed
apgd-dlr - 4/9 - 14 out of 1000 successfully perturbed
apgd-dlr - 5/9 - 5 out of 1000 successfully perturbed
apgd-dlr - 6/9 - 5 out of 1000 successfully perturbed
apgd-dlr - 7/9 - 4 out of 1000 successfully perturbed
apgd-dlr - 8/9 - 8 out of 1000 successfully perturbed
apgd-dlr - 9/9 - 10 out of 960 successfully perturbed
robust accuracy after APGD-DLR: 88.69% (total time 105.3 s)
fab - 1/9 - 0 out of 1000 successfully perturbed
fab - 2/9 - 1 out of 1000 successfully perturbed
fab - 3/9 - 2 out of 1000 successfully perturbed
fab - 4/9 - 3 out of 1000 successfully perturbed
fab - 5/9 - 0 out of 1000 successfully perturbed
fab - 6/9 - 3 out of 1000 successfully perturbed
fab - 7/9 - 1 out of 1000 successfully perturbed
fab - 8/9 - 0 out of 1000 successfully perturbed
fab - 9/9 - 2 out of 869 successfully perturbed
robust accuracy after FAB: 88.57% (total time 492.1 s)
square - 1/9 - 31 out of 1000 successfully perturbed
square - 2/9 - 36 out of 1000 successfully perturbed
square - 3/9 - 26 out of 1000 successfully perturbed
square - 4/9 - 34 out of 1000 successfully perturbed
square - 5/9 - 20 out of 1000 successfully perturbed
square - 6/9 - 15 out of 1000 successfully perturbed
square - 7/9 - 20 out of 1000 successfully perturbed
square - 8/9 - 14 out of 1000 successfully perturbed
square - 9/9 - 14 out of 857 successfully perturbed
robust accuracy after SQUARE: 86.47% (total time 727.9 s)
max Linf perturbation: 0.30000, nan in tensor: 0, max: 1.00000, min: 0.00000
robust accuracy: 86.47%

There is TF2 version's output
initial accuracy: 97.74%
apgd-ce - 1/10 - 101 out of 1000 successfully perturbed
apgd-ce - 2/10 - 127 out of 1000 successfully perturbed
apgd-ce - 3/10 - 106 out of 1000 successfully perturbed
apgd-ce - 4/10 - 106 out of 1000 successfully perturbed
apgd-ce - 5/10 - 96 out of 1000 successfully perturbed
apgd-ce - 6/10 - 73 out of 1000 successfully perturbed
apgd-ce - 7/10 - 53 out of 1000 successfully perturbed
apgd-ce - 8/10 - 55 out of 1000 successfully perturbed
apgd-ce - 9/10 - 31 out of 1000 successfully perturbed
apgd-ce - 10/10 - 76 out of 774 successfully perturbed
robust accuracy after APGD-CE: 89.50% (total time 133.7 s)
apgd-dlr - 1/9 - 8 out of 1000 successfully perturbed
apgd-dlr - 2/9 - 21 out of 1000 successfully perturbed
apgd-dlr - 3/9 - 6 out of 1000 successfully perturbed
apgd-dlr - 4/9 - 13 out of 1000 successfully perturbed
apgd-dlr - 5/9 - 4 out of 1000 successfully perturbed
apgd-dlr - 6/9 - 9 out of 1000 successfully perturbed
apgd-dlr - 7/9 - 6 out of 1000 successfully perturbed
apgd-dlr - 8/9 - 4 out of 1000 successfully perturbed
apgd-dlr - 9/9 - 7 out of 950 successfully perturbed
robust accuracy after APGD-DLR: 88.72% (total time 297.1 s)
fab - 1/9 - 0 out of 1000 successfully perturbed
fab - 2/9 - 2 out of 1000 successfully perturbed
fab - 3/9 - 1 out of 1000 successfully perturbed
fab - 4/9 - 3 out of 1000 successfully perturbed
fab - 5/9 - 0 out of 1000 successfully perturbed
fab - 6/9 - 3 out of 1000 successfully perturbed
fab - 7/9 - 1 out of 1000 successfully perturbed
fab - 8/9 - 0 out of 1000 successfully perturbed
fab - 9/9 - 2 out of 872 successfully perturbed
robust accuracy after FAB: 88.60% (total time 1808.1 s)
square - 1/9 - 28 out of 1000 successfully perturbed
square - 2/9 - 35 out of 1000 successfully perturbed
square - 3/9 - 33 out of 1000 successfully perturbed
square - 4/9 - 33 out of 1000 successfully perturbed
square - 5/9 - 21 out of 1000 successfully perturbed
square - 6/9 - 12 out of 1000 successfully perturbed
square - 7/9 - 19 out of 1000 successfully perturbed
square - 8/9 - 13 out of 1000 successfully perturbed
square - 9/9 - 18 out of 860 successfully perturbed
robust accuracy after SQUARE: 86.48% (total time 2051.4 s)
max Linf perturbation: 0.30000, nan in tensor: 0, max: 1.00000, min: 0.00000
robust accuracy: 86.48%

I finally ran the examples and everything works fine. I just noted a few small things above, and, once fixed them, I think it's ready to be merged. Do you have any idea why with TF2 is almost 3x slower than with TF1?

I fixed those bugs.
I also noticed that execution time of TF2 is much slower than that of TF1.
However, execution time is no difference when evaluating "initial accuracy".
I cannot give you a reason explain why the performance has significant gap because pytorch and TF version are totally different in my testing environment.
In my opinion, I thought attacks work properly and this PR can be merged first.

I thought pytorch side may be improved. Because the heaviest computations have done in TF side on GPU, those data may not need to transfer to GPU again in pytorch side.

Thanks for the quick updates and for the contribution!
I will investigate further the difference between the runtime in TF1 and TF2.


Manim output does not support OpenGL rendering

Manim is moving away from its Cairo renderer to using OpenGL. Some minor tweaks are needed for the Manim output for it to work properly in OpenGL mode (manim --renderer=opengl):

[x] #766
[x] #767
add_cubic_bezier_curve_to has returns unit and not an OpenGLVMobject. So It needs to do something like:wire = OpenGLVMobject()
wire.start_new_path(...)
wire.add_line_to(...)
wire.add_cubic_bezier_curve_to(...)
...

@NickHu, just to be sure, can you confirm that this is an API change in the community version of manim?

Yes:
https://github.com/ManimCommunity/manim/blob/cef3565858bcec029bbe8aa553662ade51d924eb/manim/mobject/opengl/opengl_vectorized_mobject.py#L435-L449
Although it's not clear to me if this is an intentional change or a bug.

Okok thanks for confirming.
I was thinking that actually, we may render everything as Bezier cubics (even lines with collinear controls), then just put the coordinates in a list and iterate over it to construct the path. It may make the code cleaner.

@NickHu I re-designed the manim code export to be compatible with the plethora of breaking changes they introduced in their OpenGL APi, but unfortunately it seems that their OpenGLVMobject do not support the Intersection operation, which we need for masking. While there may be other strategies for masking, they're probably very fiddly.
I propose we return to the cairo rendered export, but leave the code structured in ways to make the move to OpenGL easier in the future, when their API is a bit more mature and stable.

Since you put in the effort to switch to new API, we could keep it as an option. Maybe we could have a toggle in the export panel for choosing between Cairo and OpenGL (with an asterisk that OpenGL doesn't support braidings).

Sounds good, I'll implement it as part of #765.
@H-Yulong, could I request your help in making a checkbox in "Image export" under "Manim" with text "OpenGL renderer", default false, and expose its value in a way I can reach it from the call site of the manim::render?

Implemented fallback renderer, but it turns out they broke both APIs, no manim braidings basically never work (until we figure out an alternative strategy for it).

@regular-citizen I didn't follow this thread so not sure what's going on -- anyways, do you still want that checkbox for manim export?

Yes, sorry for closing the issue, it's still needed. The merged code now has a flag "use_opengl", which should be set to the value of the checkbox.

Let's keep track of these options in #525.


SvelteKit query-string adapter

When using query string in SvelteKit I get this error:
Avoid using history.pushState(...) and history.replaceState(...) as these will conflict with SvelteKit's router. Use the pushState and replaceState imports from $app/navigation instead.

I'm not familiar with SvelteKit, unfortunately, but it would be great, if you could contribute and make a PR with such adapter :)

It is not that simple like this. At least there should be tests and documentation. And I don't know, how to test library with SvelteKit, maybe there should be tests with some empty application build. Import from '$app/navigation' works fine inside your SvelteKit application, but I have doubts that it will work inside a library. Builders and packagers usually have different rules for your application code and code from library from node_modules folder.
Maybe it could be covered by adding support for custom history adapter 🤔


Minor bugs in validation

Codecov Report
Attention: Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.

Project coverage is 39.92%. Comparing base (e02d6c5) to head (d452fe2).
Report is 1 commits behind head on v2.

Files with missing lines
Patch %
Lines

pkg/dto/backup_policy.go
0.00%
3 Missing and 4 partials :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##               v2     #283      +/-   ##
==========================================
+ Coverage   38.28%   39.92%   +1.64%
==========================================
  Files          46       49       +3
  Lines        2641     2803     +162
==========================================
+ Hits         1011     1119     +108
- Misses       1505     1555      +50
- Partials      125      129       +4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Watch service changes to reload heuristics

WRT: CybercentreCanada/assemblyline-core#573

Codecov Report

Merging #585 (2ca7413) into master (a78569a) will increase coverage by 4.44%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #585      +/-   ##
==========================================
+ Coverage   44.55%   48.99%   +4.44%
==========================================
  Files          45       45
  Lines        6763     7387     +624
==========================================
+ Hits         3013     3619     +606
- Misses       3750     3768      +18

Impacted Files
Coverage Δ

assemblyline_core/tasking_client.py
0.00% <0.00%> (ø)

assemblyline_core/alerter/processing.py
90.21% <0.00%> (+1.08%)
:arrow_up:

assemblyline_core/dispatching/dispatcher.py
80.86% <0.00%> (+5.07%)
:arrow_up:

assemblyline_core/expiry/run_expiry.py
78.81% <0.00%> (+6.00%)
:arrow_up:

assemblyline_core/ingester/ingester.py
85.03% <0.00%> (+8.29%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a78569a...2ca7413. Read the comment docs.


Documentation for ground truth

It's unclear what the gt and uids should be in the GeneTagger example. Were these created somewhere earlier and I missed it?
Are uids created by me, or are they internal?

This is on the summer refactor warpath, closing this particular issue


Add AESGCM encrypt/decrypt functions with no dynamic checks

There was previously no easy way of accessing the AESGCM functions outside the agile EverCrypt versions. Due to modularity issues, using those functions also forced people to work in the ST effect, and not the Stack effect. This PR fixes that.

@msprotz


Add functional API (and use it from the mixin).

Examples:
import { addEventListener, runTask } from 'ember-lifeline';

...

  didInsertElement() {
    this._super(...arguments);
    addEventListener(this, fn, { passive: true });
    runTask(this, 'fooMethod', 1000);
  }

Credit to @mitchlloyd and @cibernox for suggesting this.

The main reason for this alternative API is to allow tree-shaking in the near future. I could also be more performant.

Also makes it clear where the functionality is coming from, especially from an on-boarding perspective.

Yes, completely agreed (on all accounts).

I've implemented the *Tasks portion of this issue in https://github.com/rwjblue/ember-lifeline/pull/66. I'm going to move on to create the functional equivalent for the DOM side.

Implemented in:

runTask to functional equivalent #63
*Task to functional equivalent #66
dom to functional equivalent #67

All PRs have been merged. I'll release a beta and update the docs.


Unable to parse uWSGI logs

The following should work for the sample line you posted:
log-format %^] %h %^[%^ %d %t %^] %m %U %^(%H %s) %^
date-format %b %d
time-format %H:%M:%S

Let me know how it goes.

This did not work for me. ERROR:
Parsed 1 linesproducing the following errors:

Token 'Feb' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format

My log file is in the following format:
[pid: 2398|app: 0|req: 1075/3223] <IP_ADDRESS> () {44 vars in 634 bytes} [Mon Feb 18 16:36:03 2019] DELETE /api/apps/subscription => generated 24 bytes in 10 msecs (HTTP/1.1 200) 2 headers in 71 bytes (1 switches on core 0)

Can you suggest a log format for uwsgi which would work for goaccess but still log all the details?
Originally posted by @shreyasnagare in https://github.com/allinurl/goaccess/issues/386#issuecomment-466142476

@allinurl, I tested and parser is really not possible.
I know you already suggested the same in another post, but in fact it doesn't seem to work.
And if you happen to succeed without error, the year will be so weird.

@0bi-w6n-K3nobi did you get an error on a specific token?

@allinurl, I don't remember exactly what I had done.
I believe that I declared the time-format different and gave the same error.
But, going back to this configuration, see what happens with the year:


Partially fix non-deterministic coverage

This is a band-aid fix that mostly resolves the issue with non-deterministic coverage. Without the wait, tests abruptly end and coverage can fluctuate. With the wait, Qt has time to handle its stuff for a while longer, which stabilizes the coverage. The proper fix would be to clean up everything Qt-related that is created in a test after each test, but I couldn't get that to work yet.
There are also at least two more cases of non-deterministic coverage that seem to be unrelated to Qt and which aren't fixed with this.
Related to #2302.

This PR should at least improve the situation with regard to the fluctuating coverage reports, but unfortunately it slows down the test suite a bit. A proper fix would probably include solving #2193, which lead to segfaults in my attempts so far.

I've ran the test suite 10 times and let it report coverage to coveralls. As you can see here: https://coveralls.io/builds/66882529 between the 10 reported jobs 8 had the same coverage while 2 had something different. This is an improvement over what we have on develop, where basically every run reports something different.

we seem to redefine some qapp/qtbot fixtures defined somewhere else and replace the qapp with the qtbot one? Could we add a sentence to the fixture declaration on what is going on here?
It is really not apparent to...

Yes, it is not so obvious... I'll add a comment that the qtbot fixture just re-defines the qtbot fixture from pytest-qt with additional checks done after each test.


Coursier somewhat broken on FreeBSD

Coursier does not appear to fully work on FreeBSD anymore.
The docs suggest installing a FreeBSD package. I haven't tried that because the latest version in that package manager is 1.1.0.m12, which is quite old.
There is an "Any" platform launcher, linked to https://github.com/coursier/coursier/raw/gh-pages/coursier. Downloading this (as cs) and trying to set up leads to issues:
➜  ~ ./cs setup
Exception in thread "main" java.lang.Exception: Unrecognized OS: freebsd
	at coursier.jvm.JvmIndex$.defaultOs(JvmIndex.scala:199)
	at coursier.jvm.JvmCache$.apply(JvmCache.scala:23)
	at coursier.cli.jvm.SharedJavaParams.jvmCacheOf$1(SharedJavaParams.scala:32)
	at coursier.cli.jvm.SharedJavaParams.cacheAndHome(SharedJavaParams.scala:40)
	at coursier.cli.jvm.SharedJavaParams.javaHome(SharedJavaParams.scala:61)
	at coursier.cli.setup.Setup$.run(Setup.scala:32)
	at coursier.cli.Coursier$.$anonfun$runA$2(Coursier.scala:166)
	at coursier.cli.Coursier$.$anonfun$runA$2$adapted(Coursier.scala:140)
	at coursier.cli.CommandAppPreA.run(CommandAppPreA.scala:22)
	at caseapp.core.app.CommandAppWithPreCommand.$anonfun$main$5(CommandAppWithPreCommand.scala:99)
	at caseapp.core.app.CommandAppWithPreCommand.$anonfun$main$5$adapted(CommandAppWithPreCommand.scala:99)
	at scala.util.Either.fold(Either.scala:191)
	at caseapp.core.app.CommandAppWithPreCommand.$anonfun$main$3(CommandAppWithPreCommand.scala:99)
	at caseapp.core.app.CommandAppWithPreCommand.$anonfun$main$3$adapted(CommandAppWithPreCommand.scala:85)
	at scala.Option.foreach(Option.scala:407)
	at caseapp.core.app.CommandAppWithPreCommand.main(CommandAppWithPreCommand.scala:85)
	at coursier.cli.Coursier$.main(Coursier.scala:105)
	at coursier.cli.Coursier.main(Coursier.scala)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at coursier.bootstrap.launcher.a.a(Unknown Source)
	at coursier.bootstrap.launcher.Launcher.main(Unknown Source)
➜  ~

This was after running it once with no arguments, during which it did download various dependencies. I'm using OpenJDK 18.0.1. Some other subcommands (java, java-home) yield the same failure point.
But it's not totally broken. I did ./cs install ammonite and it appears to have given me a working Ammonite shell with Scala 2.13.8 (I only tested a couple simple expressions though).
I'm happy to kick the tires on any possible fixes, including trying to fix it myself if a maintainer thinks it would likely be "small".

@csgordon I wouldn't expect cs setup to work out-of-the-box on FreeBSD. It tries to install a JVM itself if none is found (using the coursier JVM index), but this index has no JVMs for FreeBSD. It seems JVM vendors don't package JVMs for FreeBSD. My understanding is that JVMs should be installed via the FreeBSD package manager there. So on FreeBSD, coursier shoudn't try to install one. It should only complain if it doesn't find one, and maybe suggest users to install one via the FreeBSD package manager.
That said, it seems you ran the JVM launcher of coursier on FreeBSD (so I assume you already installed a JVM yourself), and apparently, coursier tried to look for a JVM in the index anyway. I'm not sure why it does that, but I guess it shouldn't, so that would need to be addressed too.


Slow Image Publishing

[X] I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stack Overflow. For library-specific questions I would look for help in the corresponding library forums.
[X] I have thoroughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

I have a question!

[X ] I searched the Wiki, open and closed issues for an answer. I tried my best to find the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:
I've been using ROS-sharp to publish out images from Unity3D into ROS.
Unfortunately, the framerates I achieve are very low.
I tried changing to raw images instead of compressed, but this made the problem worse.
Did some timing, and the jpeg compression was fast for my images (~10ms), so looked for what was causing the delay.
I've tracked down the main culprit to the WebSocket.SendAsync(data, null);
This appears to be the same problem in this issue: https://github.com/sta/websocket-sharp/issues/471
What can I do to increase framerates for images going into ROS? As is, for 1280x720 images I can't get more than ~15fps.

Thank you!

Hi @jpapon,
I did few experiments myself, and here I my observations and thoughts on this issue.
In order to make a fair judgement on the performance of the websocket communication, I first checked the frame rate of the Unity application itself, then I checked the frequency of received ROS messages via 'rostopic hz /image_topic'.
So, on my pc the Unity application runs at ~29 fps when I have an additional camera of resolution 1280x720 in the scene, of which the frames are then published as compressed images using the default ImagePublisher of the repository. On the ROS side, the frequency of the received messages are observed as ~24 fps.
When the resolution of the camera is 1920x1080, the numbers are ~14fps and ~13 fps respectively.
In the aforementioned issue, it is said that the application runs at 30 Hz, yet the the messages are published at 5 Hz.
Are you sure that your Unity application itself is running at a relatively higher fps, and yet msg publishing is only at ~15fps?
The aforementioned issue refers to the data of size ~320Kb, whereas the compressed image of 1280x720 is encoded to a msg of size ~20Kb in my case. I think the biggest bottleneck is rather the rendering of the camera as far as I can observe from the Unity profiler tool. On my pc the camera rendering takes ~21 ms, the compression takes ~10 ms.

I experimented one more setup. Instead of rendering/publishing the 1280x720 images at every Unity frame, I did split the images into two chunks of size 640x720, and rendered/published each of the chunks alternately at one Unity frame. So the Unity application framerate increased to ~ 60 fps, and the rate of receiving messages on the ROS side increased to ~48 fps.
The aim was to reduce the amount of data to be transmitted, and see its effect on the websocket communication speed.
The observation that the framerates are halved when the data size is doubled, makes me guess that websocket communication does not create an additional communication bottleneck with such small data sizes.
PS: I simply changed the rect, and the texture2D definitions dynamically. You can split an image to be rendered/published into any amount of chunks, which are to be merged back to a single image on the receiver side with an additional cost of course.

Hey @berkayalpcakal how do you have your websocket configured?
In my experiments I was just using an empty scene, so the rendering time was negligible. If I turn off the image publisher, the scene runs at 800fps. I timed the compression call in the script, and found it to be ~10 ms for 1280x720, and ~20ms for 1920x1080. I also modified the script to publish uncompressed images as well to eliminate compression time entirely.
In all cases, there was an additional substantial chunk of time spent on the CPU which limits the framerate - as far as I can tell, this was the websocket call.

outdated / closing due to inactivity


Empty reply from server on port 23763

Hello,
I succeeded in starting my Spark application with SparkLint as an extraListener but I'm getting no response from the SparkLint web server on port 23763.
I get the same issue in both yarn-client and yarn-cluster modes.
Any idea what is going wrong?
NB: I started the Spark app with following parameters:
spark-submit ...
--jars sparklint/target/jar-with-dependencies-sparklint-1.0.3-SNAPSHOT.jar \
--conf spark.extraListeners=com.groupon.sparklint.SparklintListener \
...

Hi @landier in yarn-cluster, you'll need to find out your driver node's address (lets say it is worker123.example.com) and visit worker123.example.com:23763. The stdout of your drivernode log should have some sparklint log inside to verify the port it actually uses.  In yarn-client, you'll need to hit the job submitter box's port 23763. Are these the cases you've tried?
Could you verify that you can find your app's stdout log (with sparklint content inside) on driver node?
Another situation could be some other task is occupying port 23763, so Sparklint will go with the next port 23764 instead.

Hello @roboxue ,
Yes these are the cases I tried. I can see that the Sparklint listener is registered when the application start from the application driver stdout. I could see that the port 23764 was being opened from the stdout as well from a netstat command.
The issue is really that no byte is being sent out from the Sparklint web server for us. :/

I think this could be related to interface/port bindings. The fact that logs are displaying and without exceptions, and no logs are displayed showing incoming traffic means there likely exists a network config issue so the server is not exposed to external network.
Could you try curl localhost:[port] on the job submitting box during yarn-client mode? Do that work?
Here is the code that server will log every incoming requests as the first thing to do for each request:
https://github.com/groupon/sparklint/blob/develop/src/main/scala/com/groupon/sparklint/server/RoutingMap.scala#L35
This is the code I used to instruct the server listen to <IP_ADDRESS>:[port], basically it means every network interface on the machine. Try change it to other IP (like the host's actual IP), and see if it solves your problem
https://github.com/groupon/sparklint/blob/develop/src/main/scala/com/groupon/sparklint/server/AdhocServer.scala#L80


Deserialization problem with Spring Cloud Stream & Spring Cloud Function

Describe the issue
Since Spring Cloud 2023.0.3, most of our Spring Cloud Stream bindings are failing to deserialize JSON objects.
It seems Jackson property DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES is now enabled by default.
It wasn't the case in 2023.0.2.
If I send in my message queue a JSON object with more fields than the DTO knows about : it fails during deserialization.
To Reproduce
Here is a sample repo where problem is shown.
There are 2 branches with each Spring Cloud version.
You can execute MessagingStreamsIT.
Every test runs fine in 2023.0.2 branch, while it fails in 2023.0.3.
As it's shown in some tests, adding @JsonIgnoreProperties(ignoreUnknown = true) on every DTO object seems to be a workaround.

Seems like PR https://github.com/spring-cloud/spring-cloud-function/pull/1162 addressed the same issue in a diffrent way too

This fix doesn't resolve the problem of not all modules being registered. For instance the Kotlin-Module is not registered and that leads to problems when using Kotlin (data classes).

Can you please let us know when you plan to release 4.1.4? we are waiting for this fix. We need a CVE fix from a previous hotfix but we cannot upgrade to 4.1.3 due to this issue.


Analyzer doesn't give unnecessary_import feedback if element used only from 1 export

I'm doing a barrel export and notice when using only elements from 1 export files it doesn't give unnecessary_import at all.
For example if I have
a.dart
class A {}

class A2 {}

b.dart
class B {}

c.dart
class C {}

alphabet.dart
export 'a.dart';
export 'b.dart';
export 'c.dart';

and used it like so, the analyzer give feedback unnecessary_import for all a, b, c dart files
import 'package:hmmm/src/alphabet/a.dart'; // unnecessary_import
import 'package:hmmm/src/alphabet/b.dart'; // unnecessary_import
import 'package:hmmm/src/alphabet/c.dart'; // unnecessary_import
import 'package:hmmm/src/alphabet/alphabet.dart';

void something() {
  final a = A();
  final a2 = A2();
  final b = B();
  final c = C();
}

but if only element used from a.dart, it doesn't give unnecessary_import to a, only unused_import for b and c.
import 'package:hmmm/src/alphabet/a.dart';
import 'package:hmmm/src/alphabet/b.dart'; // unused_import
import 'package:hmmm/src/alphabet/c.dart'; // unused_import
import 'package:hmmm/src/alphabet/alphabet.dart';

void something() {
  final a = A();
  final a2 = A2();
  // final b = B();
  // final c = C();
}

dart --version
Dart SDK version: 2.15.1 (stable) (Tue Dec 14 13:32:21 2021 +0100) on "windows_x64"
IDE
vscode

@srawlins

In the second case you describe, alphabet.dart and a.dart provide the same set of "used" elements (A and A2). We consider these "redundant" imports, rather than "unnecessary."
We have not yet landed the check for redundant imports. The check is more complicated, because when two import directives provide the same set of used elements, we want to be careful about recommending which import directive to remove.

Follow support for redundant imports here: https://github.com/dart-lang/sdk/issues/44569.


[BUG] Plugin SVRESYNC fails !!

Bug description:

When the PCR is included in an independent pid of a service (for example, when removing the video of a tv service to convert it to radio, leaving the original pid with only the PCR timestamps). In this case, applying the svresync plugin to this service, the result is buggy.
 But adjusted 0 PCR, 43 PTS, 0 DTS instead of adjusted 96 PCR, 126 PTS, 72 DTS.
Errors and logs:

[be sure to surround verbatim log lines with "~~~" lines]

I feel all is well expressed before.
Environment:

OS:  Windows
OS version:  Windows 10 64bit
TSDuck full version:  3.29-2651
Installation type:  PORTABLE

Additional information:

Identical behaviour in Linux.

Hi @lelegard ,
Any reason not to comment on this bug?
Here more information using the last version with the timestamps values in the analyze plugin (#1096):
Test A: OK (initial analysis)
tsp.exe -v \
 -I file test-001.ts \
 -P zap -s --eit 8810 8802 \
 -P analyze -w \
 -O drop

|============================================================================================|
|  PID: 0x00D2 (210)                                                              AVC video  |
|  PES stream id: 0xE0 (Video 0)                                                             |
|  1920x1080, high profile, level 4.0, 4:2:0                                                 |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ...... 3,072,606 b/s  Packets: .............. 7,173  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ............... 185  Unexpect: .............. 1  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ..................... 97  Packets: .............. 84  |
|                                 TSrate: ...... 47,782,780 b/s  Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|  PCR: ...... 1,722,694,546,453  PTS: .......... 5,742,376,801  DTS: ....... 5,742,369,601  |
|  to: ....... 1,722,785,272,472  to: ........... 5,742,686,401  to: ........ 5,742,668,401  |
|============================================================================================|
|  PID: 0x00DD (221)                                                       AC-3 Audio (fra)  |
|  PES stream id: 0xBD (Private stream 1)                                                    |
|  AC-3, stereo (L,R), @48,000 Hz, complete main, bsid 6                                     |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ........ 191,476 b/s  Packets: ................ 447  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ................ 43  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ...................... 0  Packets: .............. 22  |
|                                                                Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|                                 PTS: .......... 5,742,318,223                              |
|                                 to: ........... 5,742,620,623                              |
|============================================================================================

Test B: ERROR (troubles with some timestamps)
tsp.exe -v \
 -I file test-001.ts \
 -P zap -s --eit 8810 8802 \
 -P svresync -s 8810 8802 \
 -P analyze -w \
 -O drop

|============================================================================================|
|  PID: 0x00D2 (210)                                                              AVC video  |
|  PES stream id: 0xE0 (Video 0)                                                             |
|  1920x1080, high profile, level 4.0, 4:2:0                                                 |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ...... 3,072,318 b/s  Packets: .............. 7,173  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ............... 185  Unexpect: .............. 1  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ..................... 97  Packets: .............. 84  |
|                                 TSrate: ...... 47,773,759 b/s  Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|  PCR: ...... 1,722,694,546,453  PTS: .......... 5,742,376,801  DTS: ....... 5,742,369,601  |
|  to: ....... 1,191,611,917,232  to: ........... 3,972,108,572  to: ........ 3,972,090,572  |
|============================================================================================|
|  PID: 0x00DD (221)                                                       AC-3 Audio (fra)  |
|  PES stream id: 0xBD (Private stream 1)                                                    |
|  AC-3, stereo (L,R), @48,000 Hz, complete main, bsid 6                                     |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ........ 191,458 b/s  Packets: ................ 447  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ................ 43  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ...................... 0  Packets: .............. 22  |
|                                                                Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|                                 PTS: .......... 5,742,318,223                              |
|                                 to: ........... 3,972,042,799                              |
|============================================================================================|

|============================================================================================|
|  PID: 0x03F2 (1010)                                                             AVC video  |
|  Clock values range:                                                                       |
|  PCR: ...... 1,191,521,413,086  PTS: .......... 3,971,816,140  DTS: ....... 3,971,808,940  |
|  to: ....... 1,191,612,118,469  to: ........... 3,972,086,140  to: ........ 3,972,075,340  |
|============================================================================================|

Problem: The initial timestamps (PCR/PTS/DTS) seems to not be changed. I don't know when the jump is done. But the end values appear correct comparing to the pid 1010 (main clock for the target service).

 Clear  Adapt.F.: ................ 97  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  Sections:                   |
|                                 PCR: ..................... 97  Unit start: ............ 0  |
|                                 TSrate: ...... 44,651,387 b/s                              |
|  Clock values range:                                                                       |
|  PCR: ...... 1,722,694,546,453                                                             |
|  to: ....... 1,722,785,272,472                                                             |
|============================================================================================|
|  PID: 0x00DD (221)                                                       AC-3 Audio (fra)  |
|  PES stream id: 0xBD (Private stream 1)                                                    |
|  AC-3, stereo (L,R), @48,000 Hz, complete main, bsid 6                                     |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ........ 195,676 b/s  Packets: ................ 447  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ................ 43  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ...................... 0  Packets: .............. 22  |
|                                                                Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|                                 PTS: .......... 5,742,318,223                              |
|                                 to: ........... 5,742,620,623                              |
|============================================================================================|

Explanation: The craft of the PCR pid in the service is done right. The same number (97) of PCR packets exist and with correct values. Only the PTS and DTS timestamps of the video (that is removed) are missing. So no problem at all.

 Clear  Adapt.F.: ................ 97  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  Sections:                   |
|                                 PCR: ..................... 97  Unit start: ............ 0  |
|                                 TSrate: ...... 44,651,387 b/s                              |
|  Clock values range:                                                                       |
|  PCR: ...... 1,722,694,546,453                                                             |
|  to: ....... 1,722,785,272,472                                                             |
|============================================================================================|
|  PID: 0x00DD (221)                                                       AC-3 Audio (fra)  |
|  PES stream id: 0xBD (Private stream 1)                                                    |
|  AC-3, stereo (L,R), @48,000 Hz, complete main, bsid 6                                     |
|  Service: 0x2262 (8802) CANAL+ DECALE                                                      |
|--------------------------------------------------------------------------------------------|
|  Single Service PID             Transport:                     Discontinuities:            |
|  Bitrate: ........ 195,676 b/s  Packets: ................ 447  Expected: .............. 0  |
|  Access: ............... Clear  Adapt.F.: ................ 43  Unexpect: .............. 0  |
|                                 Duplicated: ............... 0  PES:                        |
|                                 PCR: ...................... 0  Packets: .............. 22  |
|                                                                Inv.Start: ............. 0  |
|  Clock values range:                                                                       |
|                                 PTS: .......... 5,742,318,223                              |
|                                 to: ........... 5,742,620,623                              |
|============================================================================================|

Problem: None of the timestamps marks have been changed. Neither those of PCR nor those of PTS. So the plugin fails to apply it when the PCR is in a independent pid with only PCR values.

Please can you check this?
Thank you.

Hi @lelegard ,
Regarding the current troubles with the svresync plugin, Iet to explain two topics:

I know that the TSDuck is a realtime tool. So it can't change the time (someone says: back to the future?). Then the outcomes from the current implementation regarding the initial not changed timestamps could be considered correct (Test C). That's because until the plugin doesn't have the timestamps of the reference pid then it's impossible to calculate any new values. And I agree with this. However, the current implementation lacks a method to discard these uncheangable timestamps. Let me to explain more: if you need to "resync" some service that after you want to remux, then the "non-recomputed" timestamps generate troubles in the muxer. So my suggestion is to add one new parameter to the plugin like -P svresync --mute-pid value ... which indicates that the plugin will remove the pid value until you have the required information to change the timestamps of this pid. Because, yes the plugin can't know the pids of the services until it reads them, and it can't know the target time value to recalculate the timestamps after have all the information. But until this point of the time, if the user knows the involved pids and can pass them as a parameters, then you could mute these pids until this time. Don't you think so?
The other issue is created by a simple edge case: when the PCR timestamps are inserted in a PID that doesn't includes any PES data (check the PCR (not otherwise referenced) message). The plugin seems that in this case it refuses to change the timestamps because it doesn't agree with the PMT information. However, this is a valid stream. Therefore I suggest to remove this limitation. You agree?

I hope you want to target this issue soon.
Regards.

Hi @lelegard ,
Regarding the second issue (failure on Test D because the empty PCR pid) this patch resolves the issue:
diff --git a/src/tsplugins/tsplugin_svresync.cpp b/src/tsplugins/tsplugin_svresync.cpp
index 0212034..2493a52 100644
--- a/src/tsplugins/tsplugin_svresync.cpp
+++ b/src/tsplugins/tsplugin_svresync.cpp
@@ -229,7 +229,7 @@ ts::ProcessorPlugin::Status ts::SVResyncPlugin::processPacket(TSPacket& pkt, TSP
     }

     // Adjust time stamps in the target service (if we have a reference).
-    if (_last_ref_pcr != INVALID_PCR && _target_pids.test(pkt.getPID())) {
+    if (_last_ref_pcr != INVALID_PCR && (_target_pids.test(pkt.getPID()) || _target_pcr_pid == pkt.getPID())) {

         // If the target packet contains a PCR, adjust the time difference between the two services.
         if (pkt.hasPCR()) {

So, please merge it. However, I'm unsure if this edge case could be present on other parts of the toolkit. So perhaps it will be necessary to review the PIDSet::test() function to consider this case. What you think?

Hi @lelegard ,
Regarding the edge case related to pids with only PCR timestamps and no PES content, the filter rmorphan has the same behaviour. I feel the PMT class requires a review to include such pids. You agree with that?

Hi @lelegard ,
Regarding the enhancement for the non_changed_timestamps I've another proposal different to the --mute-pid value. I feel this suggestion will be more close to your idea of the KISS model and the current implementation:

Add to the svresync plugin the parameter --touch-label-pid 1:100,101,... : the concept is very easy. When the changing of the timestamps starts to apply inside a pid (i.e. 100 or 101 or etc.), the label flag indicated (i.e 1) is then set for all the rest of packets in this pid.

I believe you will quickly understand the utility of this. Does this solution look simple and clean? What do you think?

Hi
I am going to merge #1116, thank you.

Regarding the edge case related to pids with only PCR timestamps and no PES content, the filter rmorphan has the same behaviour.

I don't think so. The PCR PID of each service is marked as referenced in rmorphan.
Referencing an independent PID as PCR PID in a PMT is well-known and usually handled properly. Except in svresync where it was forgotten. It is possible that other omissions exist but this is not a general problem and must be investigated case by case.

I feel the PMT class requires a review to include such pids.

The PMT class simply describes a PMT. Properly using its pcr_pid is an application issue and does not impact the PMT class itself.

I am going to merge #1116, thank you.

You're welcome.

Regarding the edge case related to pids with only PCR timestamps and no PES content, the filter rmorphan has the same behaviour.

I don't think so. The PCR PID of each service is marked as referenced in rmorphan.

Right. I've check it after commenting it. So don't worry at all.

I feel the PMT class requires a review to include such pids.

The PMT class simply describes a PMT. Properly using its pcr_pid is an application issue and does not impact the PMT class itself.

Yes, another time. Handling the PMT::pcr_pid it's sufficient to overcome this edge case in any circunstance. If I encounter a similar problem, then I'll report it. Thank you for your support.


Upgrade Google Java Format 1.22.0 -> 1.23.0

This PR contains the following updates:

Package
Type
Update
Change

Google Java Format
compile
minor
1.22.0 -> 1.23.0

Release Notes

google/google-java-format (Google Java Format)
v1.23.0
Compare Source
Changes:

Adjust indentation of line comments inside expression switches (#876)
Fix modifier order handling for non-sealed (https://github.com/google/google-java-format/commit/7fd9300499228717bcfee75ff4f836507df832ee)
Handle empty classes with no members that contain semi-colons (https://github.com/google/google-java-format/commit/b9b6730b80a16798da559f2d50112cec3d8538ad)
Support overlapping ranges in --lines (https://github.com/google/google-java-format/commit/c4b467d50bd6bb52935075b438f602ff902db167)
Remove support for the String Templates preview feature, see https://bugs.openjdk.org/browse/JDK-8329949 (https://github.com/google/google-java-format/commit/fdf4b29a6e1206f451bd96ed7967f4061921881a)
Improve compatibility with latest JDK early access builds (https://github.com/google/google-java-format/commit/0b384e32b39117a8da2829d8543e7474338ddebe)

Full Changelog: https://github.com/google/google-java-format/compare/v1.22.0...v1.23.0

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade Google Java Format 1.22.0 -> 1.23.0

See:
- https://github.com/google/google-java-format/releases/tag/v1.23.0
- https://github.com/google/google-java-format/compare/v1.22.0...v1.23.0


[Bug]: darkmode error

Reproduction
<script setup lang="ts">
import { useColorMode,useDark } from '@vueuse/core'
// import { Icon } from '@iconify/vue'
import { Button } from '@/components/ui/button'
import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigger } from '@/components/ui/dropdown-menu'

const mode = useColorMode()
const isDark = useDark()
</script>

<template>
  <DropdownMenu>
    <DropdownMenuTrigger as-child>
      <Button variant="outline">
        <span class="sr-only">Toggle theme 》》{{isDark}}《《 </span>
      </Button>
    </DropdownMenuTrigger>
    <DropdownMenuContent align="end">
      <DropdownMenuItem @click="mode = 'light'">
        Light
      </DropdownMenuItem>
      <DropdownMenuItem @click="mode = 'dark'">
        Dark
      </DropdownMenuItem>
      <DropdownMenuItem @click="mode = 'auto'">
        System
      </DropdownMenuItem>
    </DropdownMenuContent>
  </DropdownMenu>
</template>

Describe the bug
I'm developing using vue3+vite+ts. After installing vite according to the Installation guide, I proceeded with implementing dark mode. Upon clicking 'toggle theme', the HTML root element toggles its class between 'light' and 'dark'. However, the page itself doesn't switch to 'light' or 'dark', nor does it modify '@media (prefers-color-scheme: light)'.
When I switch between light and dark modes using the appearance settings on my Mac OS, it disregards the root element's class as either 'light' or 'dark'. Additionally, I've added the '@media (prefers-color-scheme: light)' CSS property. I'm wondering if the logic for dark mode in the current documentation is sufficient, or how can I need to modify my code to fix it the bug or what kind a part of the code I'm missing.
If you could take the time to help, it would be greatly appreciated!
System Info
System:
    OS: macOS 14.4.1
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Memory: 30.13 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 21.7.1 - /usr/local/bin/node
    Yarn: 1.22.11 - /usr/local/bin/yarn
    npm: 10.5.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 123.0.6312.107
    Safari: 17.4.1

Contributes

[ ] I am willing to submit a PR to fix this issue
[ ] I am willing to submit a PR with failing tests

https://github.com/radix-vue/shadcn-vue/assets/32637205/3b4ad77c-a546-4d0f-8b8e-95e6b0bf1061

check your tailwind.config.js/ts
what's the darkMode value in there.
source code would be helpful

@KelvinTee58
I checked your source
import { createApp } from "vue";
import App from "./App.vue";

// 自定义样式

// tailwindcss main css file path was wrong
import "./assets/index.css";

// with aliases
// import<EMAIL_ADDRESS>
import "./style.css";

const app = createApp(App);

// 路由
import router from<EMAIL_ADDRESS>app.use(router);

app.mount("#app");

Your main tailwind CSS files in in src/assets/index.css but you was pointing to @/assets/styles/index.scss

Hi sadeghbarati.
The problem has been resolved. Thank you for your help.


Dockerfile issue

RUN apt-get install --no-install-recommends -y curl=7.74.0-1.3+b1
should update to
RUN apt-get install --no-install-recommends -y curl=7.74.0-1.3+deb11u1

Hello @drabdrab 👋
Thanks a lot for your report and pointing out the correction 🙏
Pull request is open and waiting for approval #53

@drabdrab correction was merged and zenika/terraform-aws-cli:latest image tag is available for you to test !


day13

Codecov Report

Merging #60 (b27083f) into day13 (d406a0c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             day13       #60   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           92        92
  Branches         5         5
=========================================
  Hits            92        92

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


TypeError: Object is not a function issue when Object message exists in proto file

Hello everyone, and thank you for this library!
I have encountered an issue with a proto file that includes an "Object" message (yes, I know that is not a good name for the message, but it is what it is). For the sake of simplicity, it looks like this:
syntax = "proto3";

message Object {
     string object_id = 1;
}

message ObjectList {
    repeated Object object = 1;
}

message RequestMessage {
}

service TestService {
    rpc Get(RequestMessage) returns (Object);
    rpc GetAll(RequestMessage) returns (ObjectList);
}

When I run my React app which is build with CRA I got this error:

Any ideas how to resolve the issue? Except message renaming :)
I did not have this problem using protoc-gen-grpc-web protoc plugin.

Can you provide more details for how to reproduce this issue (protoc args, tsconfig, etc.)? As far as I know name clashes with global/reserved identifiers is already taken care of and it's totally valid to have an export like: export const Object = new Object$Type(); and import it in other files
import { MessageType } from '@protobuf-ts/runtime';
import { Object } from '../example';

console.log('Object instanceof MessageType', Object instanceof MessageType);
console.log(Object.create());

@jcready Thank you for your reply. I have this issue in my old project which is quite complex. I will try to create a small demo and let you know.

Also, if you could show what the "View compiled" (at the bottom of your screenshot) shows that would also help. FWIW it seems like the reported error message shown isn't lining up correctly (e.g. imperfect sourcemap mapping) based on the fact that the line shown doesn't use a runtime Object identifier at all.

I was able to reproduce the issue. You can find the application code here:
https://github.com/larscliff/protobuf-ts-object-issue

So far, I've found two ways to work around the problem. Either update the React version to 18 or remove ">0.2%" from the development section in package.json.
I don't know, perhaps there's a better solution?

I was able to reproduce the issue. You can find the application code here:
https://github.com/larscliff/protobuf-ts-object-issue

So far, I've found two ways to work around the problem. Either update the React version to 18 or remove ">0.2%" from the development section in package.json.
I don't know, perhaps there's a better solution?

Yeah, it looks like a bug with webpack where it seems to incorrectly assume that the in-scope Object identifier will always be referencing the global Object class when that isn't the case.
It looks like you might be able to adjust the webpack config to disable the Object(...) wrapping which is the source of the issue: https://github.com/webpack/webpack/issues/5600#issuecomment-368333132

Thank you, I will try.


Added legacy AdMob SDK back into GMA feature branch.

Description

Provide details of the change, and generalize the change in the PR title above.

Adds the older AdMob SDK back into the source base.  This way we can have a longer deprecation window.
Updates the way the AdMob C++ SDK for iOS tracks header dependencies when building its framework.  Our build system doesn't allow us to be dependent on two different versions of the GMA iOS SDK pods (1: older Firebase AdMob pinned version and 2: the new Firebase GMA SDK which will track new iOS GMA SDK releases). Therefore,

the AdMob SDK is now tied to a pinned version of headers which have been checked-in to this admob/admob_resources/ios_headers, and
the new Google Mobile Ads (GMA) SDK uses the standardios_pod/Podfile to track the latest and greatest iOS GMA SDK.

removed tools/ios/tetapp from both admob/ and gma/.  Not quite sure why it was migrated from G3.

Note: the linter complains about some of the formatting in the iOS headers in the podfile.  I tried to fix the problems, but then the formatter puts them back.  I'm going with the formatter's call on thsi one.
Changed files of note:

admob/admob_resources/ios_headers/

contains the GMA SDK 7.69 header files.

CMakeLists.txt

reinstated add_subdirectory(admob)

app/CMakeLists.txt

reinstated admob_HDRS

build_scripts/ios/build.sh

reinstated firebase_admob

scripts/gha/build_ios_tvos.py

reinstated firebase_admob

Testing

Describe how you've tested these changes. Link any manually triggered Integration tests or CPP binary SDK Packaging Github Action workflows, if applicable.

Integration Test CI for AdMob and GMA targets
Packaging CI

Type of Change
Place an x the applicable box:

[ ] Bug fix. Add the issue # below if applicable.
[ ] New feature. A non-breaking change which adds functionality.
[X] Other, such as a build process or documentation change.

FYI we can add a .clang-format-ignore file and have it ignore that whole directory. This way you can just use the files as-is.
I also think there is a CPPLINT.cfg file we can put into that directory and set it to exclude all the files.

FYI we can add a .clang-format-ignore file and have it ignore that whole directory. This way you can just use the files as-is.
I also think there is a CPPLINT.cfg file we can put into that directory and set it to exclude all the files.

Ok. Well the files are already formatted so I didn't add a clang format ignore file.  I did add a lint suppression file, however.
Hopefully these files never change and it's not required, but it's nice not to have to dismiss the linter's comments in this PR at least. :)


Consider changing "Set Services to Manual" from checkbox to toggle, so that unsetting it, would restore the default startup for the involved services

Describe the solution you'd like
Using winutil 24.06.28.
Consider changing "Set Services to Manual" from a checkbox to a toggle, so that unsetting it, would restore the default startup for the involved services.

@jcrmatos, you can restore the changes made by this tweak by selecting it and clicking the Undo Tweaks button.

OK, thanks. Does it work the same way for all the Tweaks?

@jcrmatos Yes, it is.
The UI is a bit odd right now (buttons are unclear for new users).. so I'm currently working on changing the UI. You can follow the progress in Progress Tracking Issue https://github.com/ChrisTitusTech/winutil/issues/2226

OK, thanks.
Yes, the UI needs some love.
Should I close this issue?

OK, thanks.

No problem, I'm glad I could help 😄

Yes, the UI needs some love.
Should I close this issue?

I'll keep this suggestion og your into consideration, maybe make WinUtil Tweaks Tab be like O&O Shutup Program, but this isn't really finalized yet.
In general, thanks for sharing these suggestions of yours, @jcrmatos 👍

Thanks.


Add option to disable printing 0 bytes as ascii 0.

Consider the following output

The right column present ASCII values of the hex column.
In the case of null bytes (value 0), they are printed as ascii 0 in the right column.
This is confusing to say the least. Consider for example the case of no color (it text after all. User will copy & paste it elsewhere):

As a comparison, here is xxd output, where they denote non-printable characters with a period instead:

I did not find any command line arguments to disable this.
I would like to request a mode of hexyl to print non-printable characters in a similar manner as xxd does it. Specifically I would like it to not insert printable ascii that does not belong in the ascii printout:

0
_
•
And possibly more.

Good point, thank you. See discussion in #169


Request: OnSelect listeners for TreeView

Without the TreeView items being connectable with onselect actions, the TreeView is quite useless.
So it would be very nice to have an on_selection_listener for the TreeView just like ListView has.

Hello @saddy001 , thank you for the suggestion. However remember that you can set the onclick listener for each TreeItem.

Unfortunately this doesn't work, because onclick is already overriden on TreeItem to provide the tree expand function. So set_on_click_listener is non-functional.

Sorry, my mistake. The above is only true for TreeView, not TreeItem. Thanks.


Add systemd script to install

This also disables the upstart install that is breaking many RaspberryPis.
Closes #7

Thanks @sukima ! 👍


0x020C4300 error in operating systems made with Cosmos

So, I was creating an OS named RadonCLI (I will rename it to RadonOS soon 😜), and it worked normally on VirtualBox and VMware. I tried testing this on copy.sh, and the output is like this:

Please fix this. Click here if you want to view Cosmos repository.

Exception 6 is invalid opcode. This is because v86 is not able to do some
fancy instruction cosmos uses that is not in the Pentium 2 instruction set.
Also, please google before reporting an issue. Quick Google search
revealed cpu exception 6 is invalid opcode.
Can you provide the exact code? V86's debug logs? What instruction caused
this?
On Mon, Jul 27, 2020, 12:32 PM kilickerem066<EMAIL_ADDRESS>wrote:

So, I was creating an OS named RadonCLI (I will rename it to RadonOS
soon 😜), and it worked normally on VirtualBox and VMware. I tried
testing this on copy.sh https://copy.sh/v86/, and the output is like
this:
[image: Screenshot]
https://user-images.githubusercontent.com/64794941/88532443-493b5980-d00d-11ea-9527-52f8be16358d.png
Please fix this. Click here https://github.com/CosmosOS/Cosmos if you
want to view Cosmos repository.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/copy/v86/issues/363, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJ3IJSIGLAXEBCGNYWY7L3LR5VJT7ANCNFSM4PIT2EQA
.

In debug, this happens:

This bit is SSE enable according to wikipedia. (9th bit of cr4 does tgat)

Can you explain more?

It enables a CPU feature called SSE. (Streaming SIMD Extensions)

Basically it needs something that v86 doesn't support.

Oh.

This should be fixed now.


Adjust splunk_hec to distinguish decoded and raw bytes received

What does this PR do?
Similar to #1166 this commit adds a decoded_bytes_received so that the telemetry
from this blackhole matches the others in the project, measuring the raw/wire bytes
in bytes_received.

#1167  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Merge activity

Dec 17, 5:59 PM EST: A user merged this pull request with Graphite.


update text based emotion detection

Project-Guidance! 🎊🎈
Welcome! 🎉
Thank you for visiting our repository! We're excited to have you here. If you find this project useful, please consider giving it a star! ⭐
🎉 Have you reviewed the Contributing Guidelines? 🤔

[] YES
[x] NO

📝 Description
This proposal introduces a Text-Based Emotion Detection feature to Project-Guidance. The feature will involve preprocessing text, building a classifier to detect emotions from text, and an interactive interface for users to view results. This addition aims to enhance the project's scope and usability.
🔧 Fixes #issue_no

(Provide the specific issue number here if applicable)
🏷 Initiative Participation
This pull request is associated with the following initiative:

[x] 🌸 GSSoC (Girl Script Summer of Code)
[x] 🌼 GSSoC 2024 Extd.
[x] 🍂 Hacktoberfest 2024
[ ] ❄ Script Winter of Code (SWoC)
[ ] ⛄ Winter of Blockchain (WoB)
[x] 🙋♂ Personal
[ ] 📌 Other (please specify): _______________

@Balaji91221 KIndly mentio issue number.

@Balaji91221 KIndly mentio issue number.

This is not a issue I have added the project

@Balaji91221 You have to follow this guidelines

@Balaji91221 You have to follow this guidelines

Hi @Kushal997-das ,
I understand that the mentor of GSSoC is currently reviewing the project. I have followed the process:

Raised an issue (step 1).

My project is currently under review (step 2).

However, the review is taking longer than expected. Since there was no update, I went ahead and created a pull request. I apologize for any confusion caused by this.


openconnect-gui: Add version 1.5.3

https://openconnect.github.io/openconnect-gui/
It's a VPN client that supports Cisco AnyConnect.
It requires a TAP driver from OpenVPN. The installer does contain it in Drivers\tap-windows.exe, but version is not new. And it could conflict with openvpn, so I decided to leave a user to install it manually. The best way to do it probably, is to create a separate manifest for tap driver and require or suggest it here.
Connection profiles and settings are stored in registry. There is a INI-config support, but, if I understand correctly, it's defined during build time: https://github.com/openconnect/openconnect-gui/blob/1b9bc0ae61496a871dbee955ec2443e46d411ed4/src/main.cpp#L135.
Also it contains an openconnect cli tool.

??

I don't think it should be persisted if that's what you mean. The app doesn't touch it as far as I know.


Update the "role-based access control" link

Proposed changes

On page "https://docs.docker.com/ee/ucp/user-access/", updated the hyperlink "role-based access control" to point to "https://docs.docker.com/ee/ucp/authorization/" instead of "https://docs.docker.com/ee/access-control", because the old page does not exist anymore.
The new page seems like the most appropriate spot to find information regarding RBAC.
Related issues (optional)
Fixes #8602

Deploy preview for docsdocker ready!
Built with commit afaf74c1eceb758199abc4bdf73fbd4e30c4f21c
https://deploy-preview-8611--docsdocker.netlify.com


PowerShell 7 uses pwsh.exe

We're expecting powershell.exe but it's now  C:\Program Files\PowerShell\7\pwsh.exe.

From https://learn.microsoft.com/en-us/powershell/scripting/whats-new/differences-from-windows-powershell?view=powershell-7.4#renamed-powershellexe-to-pwshexe

Renamed powershell.exe to pwsh.exe
The binary name for PowerShell has been changed from powershell(.exe) to pwsh(.exe). This change provides a deterministic way for users to run PowerShell on machines and support side-by-side installations of Windows PowerShell and PowerShell.


'Got error No model on context stack' warning running spawning salmon notebook

Thanks for providing this code and the tutorial.
When I run the spawning_salmon.ipynb notebook I get an error/warning in the cell with the following statement:
az.plot_trace(linear_trace);

This is the error message:
Got error No model on context stack. trying to find log_likelihood in translation.
Got error No model on context stack. trying to find log_likelihood in translation.

And the plots don't look right:

Any ideas what is causing this?  I can't find any help on this error on the internet other than this post on stackoverflow which mentions it incidentally.
The results look about right:

Details:
python_version(), pm.__version__
('3.9.13', '3.11.2')


No license for this script

This script doesn't have a license in the header or the repository

Just added one, thank you


Fix badge on README

Please check if the PR fulfills these requirements

[X] The commit message follows our guidelines
[X] Tests for the changes have been added (for bug fixes / features)
[X] Documentation has been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
A fix for the non working badge on the readme
Is there an open issue that this resolves? If so, please link it here.
nope
What is the current behavior? (You can also link to an open issue here)
Badge displays error
What is the new behavior (if this is a feature change)?
Badge displays proper information
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
This pr does not affect the application
Other information:
This is a very simple pr

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@capnkenny Glad to help!


OSGi Runtime artifact does not export correct package versions

(Migrated from https://openrdf.atlassian.net/browse/SES-766)
Looking at http://repo.aduna-software.org/maven2/releases/org/openrdf/sesame/sesame-runtime-osgi/2.3.3/ (and older versions) there is no version information on package level. With this, you disable the use of importing bundles that expect a certain version (-range) like: (&(package=info.aduna.xml)(version>=2.6.0)(!(version>=3.0.0) Because this data can generated easily with bnd (and its plugin tooling) there is no reason to not fix the manifest in that way.

This is very old and we have done several major overhauls of the osgi bundle setup since then. Assuming fixed or no longer relevant.


Rewrite LimitedQueue + Related Optimizations

Description
Rewrites LimitedQueue<T> to use a circular buffer instead of allocating and moving around tons of shared_ptrs. Adds some additional features to the limited queue like find/rfind so that taking snapshots of the entire queue is less necessary. LimitedQueueSnapshot<T> has also been updated with convenience functions like begin/end for use in range-based for loops/libraries.
Attempting to profile the application on macOS with Instruments (lol) shows a minor speedup and a bit lower memory usage, but I'm not sure if I'm really using the profiler correctly. I added some benchmarks and tests, too.
This rewrite also seems to fix #1421, where Chatterino would crash on receiving a message after loading 1000 historical messages.
Disclaimer: I was not 100% sure which methods in LimitedQueue did not need to be guarded by the mutex, so please verify my decisions.

Benchmark (w/ mutex locking)
This PR (5af89c3):

Run on (8 X 24.1208 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x8)
Load Average: 4.86, 3.55, 2.54
---------------------------------------------------------------------------------
Benchmark                                       Time             CPU   Iterations
---------------------------------------------------------------------------------
BM_LimitedQueue_PushBack                     19.4 ns         19.4 ns     34152006
BM_LimitedQueue_PushFront_One                 670 ns          675 ns      1039594
BM_LimitedQueue_PushFront_Many               2834 ns         2834 ns       247147
BM_LimitedQueue_Replace                       261 ns          261 ns      2678934
BM_LimitedQueue_Snapshot                     1030 ns         1030 ns       679704
BM_LimitedQueue_Snapshot_ExpensiveCopy       3710 ns         3710 ns       188782
BM_LimitedQueue_Find                         1729 ns         1729 ns       405879

Pull request checklist:

[x] CHANGELOG.md was updated, if applicable

@pajlada your changes look great, thanks for doing the shared_mutex stuff!

Agree, can resolve the shared lock concerns if the methods were private (as the calling scope likely would've already acquired a lock)

I added most of the suggested locks. However, I'm still unsure of the necessity of locking for simple accessors like empty()  or space() that return integral types.
I was reading this answer on StackOverflow which seems to suggest it might not be needed, but I'm not very well versed with multi-threaded programming so I still want more discussion around it.

the safest approach is to lock or privatize
in the public + no lock scenario,

to avoid undefined behavior with a concurrent read/write, you'd want the int to be updated atomically. since std::atomic isn't explicitly being used, the atomicity depends on the underlying architecture (and I would prefer the variable marked as volatile to avoid compiler optimizations that could get in the way of atomicity, but that's not possible here)
further, possible memory barriers could make concurrent reads on different threads yield different values, even if there is no concurrent write (though this degree of correctness probably isn't strictly required for this application)

I added two more accessors, first and last which are the get equivalent of front and back. Adding the two made me wonder whether we should even bother supporting unchecked access of elements because it seems hard for callers to make guarantees about the size/emptiness of the queue.
We could just have front, back, and at which all return boost::optional<T> instead of having two for each type of access.

Adding the two made me wonder whether we should even bother supporting unchecked access of elements because it seems hard for callers to make guarantees about the size/emptiness of the queue.

Please correct me if I'm wrong but in strictly single-threaded use, callers should be able to make such guarantees, right?

Please correct me if I'm wrong but in strictly single-threaded use, callers should be able to make such guarantees, right?

that's correct (no issues if all reads/writes done from the same thread)


Chebato/setup phase

Added the setup phase test suite

In the resourceSelectedCards the resourceCard method should be player.resourceCard(card, false); to ensure that the cards are ready when resource and available on the first turn. The test should check for this as well.

In the ResourcePrompt the resourceSelectedCards the resourceCard method should be player.resourceCard(card, false); to ensure that the cards are ready when resource and available on the first turn. The test should check for this as well.

I changed the player.resourceCard(card) -> player.resourceCard(card, false); and added tests to verify they are indeed unexhausted.


Crashing Ecto/Postgrex

I'm working on an Elixir application and I've managed to create a scenario that causes Ecto to crash.
It seems like this causes Ecto to crash with three different types of errors.
I've created a similarish app to reproduce the scenario. See ecto-crash
To reproduce please run the following commands:
$ git clone https://github.com/walkr/ecto-crash
$ cd ecto-crash
$ mix deps.get
$ make crash  # See Makefile for details

NOTE: Run make crash repeatedly and you will get different errors. See bellow under Current Behavior
Here's my database config:
config :ecto_crash, EctoCrash.Repo,
  adapter: Ecto.Adapters.Postgres,
  extensions: [],
  username: "postgres",
  password: "postgres",
  database: "ecto_crash",
  hostname: "localhost",
  pool_size: 16,
  pool_timeout: 10_000,
  timeout: 10_000

config :ecto_crash, ecto_repos: [EctoCrash.Repo]

Environment

Elixir version (elixir -v):
Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false [dtrace]
Elixir 1.3.4

Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.):
postgres (PostgreSQL) 9.5.4

Ecto version (mix deps):

connection 1.0.4 (Hex package) (mix)
poolboy 1.5.1 (Hex package) (rebar)
decimal 1.2.0 (Hex package) (mix)
db_connection 1.0.0 (Hex package) (mix)
postgrex 0.12.1 (Hex package) (mix)
ecto 2.0.5 (Hex package) (mix)

Operating system:
ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G1108

Current behavior
The following errors are generated:
ERROR 1

  1) test insertion (EctoCrashTest)
     test/ecto_crash_test.exs:12
     ** (DBConnection.ConnectionError) tcp recv: closed
     stacktrace:
       lib/ecto/adapters/postgres/connection.ex:97: Ecto.Adapters.Postgres.Connection.execute/4
       lib/ecto/adapters/sql.ex:235: Ecto.Adapters.SQL.sql_call/6
       lib/ecto/adapters/sql.ex:454: Ecto.Adapters.SQL.struct/6
       lib/ecto/repo/schema.ex:397: Ecto.Repo.Schema.apply/4
       lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
       lib/ecto/repo/schema.ex:124: Ecto.Repo.Schema.insert!/4
       (ecto_crash) lib/ecto_crash.ex:28: EctoCrash.create_user/1
       test/ecto_crash_test.exs:14: anonymous fn/2 in EctoCrashTest.test insertion/1
       (elixir) lib/enum.ex:1628: anonymous fn/3 in Enum.reduce/3
       (elixir) lib/range.ex:92: Enumerable.Range.reduce/5
       (elixir) lib/enum.ex:1627: Enum.reduce/3
       test/ecto_crash_test.exs:13: (test)

ERROR 2

  1) test insertion (EctoCrashTest)
     test/ecto_crash_test.exs:12
     ** (DBConnection.ConnectionError) tcp recv: invalid argument - :einval
     stacktrace:
       lib/ecto/adapters/postgres/connection.ex:97: Ecto.Adapters.Postgres.Connection.execute/4
       lib/ecto/adapters/sql.ex:235: Ecto.Adapters.SQL.sql_call/6
       lib/ecto/adapters/sql.ex:454: Ecto.Adapters.SQL.struct/6
       lib/ecto/repo/schema.ex:397: Ecto.Repo.Schema.apply/4
       lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
       lib/ecto/repo/schema.ex:124: Ecto.Repo.Schema.insert!/4
       (ecto_crash) lib/ecto_crash.ex:28: EctoCrash.create_user/1
       test/ecto_crash_test.exs:14: anonymous fn/2 in EctoCrashTest.test insertion/1
       (elixir) lib/enum.ex:1628: anonymous fn/3 in Enum.reduce/3
       (elixir) lib/range.ex:92: Enumerable.Range.reduce/5
       (elixir) lib/enum.ex:1627: Enum.reduce/3
       test/ecto_crash_test.exs:13: (test)

ERROR 3
  1) test insertion (EctoCrashTest)
     test/ecto_crash_test.exs:12
     ** (exit) exited in: :gen_server.call(EctoCrash.Repo.Pool, {:checkout, #Reference<<IP_ADDRESS>>, true}, 10000)
         ** (EXIT) shutdown
     stacktrace:
       :erlang.send(EctoCrash.Repo.Pool, {:"$gen_cast", {:cancel_waiting, #Reference<<IP_ADDRESS>>}}, [:noconnect])
       (stdlib) gen_server.erl:402: :gen_server.do_send/2
       (stdlib) gen_server.erl:232: :gen_server.do_cast/2
       src/poolboy.erl:58: :poolboy.checkout/3
       (db_connection) lib/db_connection/poolboy.ex:41: DBConnection.Poolboy.checkout/2
       (db_connection) lib/db_connection.ex:712: DBConnection.checkout/2
       (db_connection) lib/db_connection.ex:619: DBConnection.run/3
       (db_connection) lib/db_connection.ex:463: DBConnection.prepare_execute/4
       lib/ecto/adapters/postgres/connection.ex:91: Ecto.Adapters.Postgres.Connection.execute/4
       lib/ecto/adapters/sql.ex:235: Ecto.Adapters.SQL.sql_call/6
       lib/ecto/adapters/sql.ex:454: Ecto.Adapters.SQL.struct/6
       lib/ecto/repo/schema.ex:397: Ecto.Repo.Schema.apply/4
       lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
       lib/ecto/repo/schema.ex:124: Ecto.Repo.Schema.insert!/4
       (ecto_crash) lib/ecto_crash.ex:28: EctoCrash.create_user/1
       test/ecto_crash_test.exs:14: anonymous fn/2 in EctoCrashTest.test insertion/1
       (elixir) lib/enum.ex:1628: anonymous fn/3 in Enum.reduce/3
       (elixir) lib/range.ex:92: Enumerable.Range.reduce/5
       (elixir) lib/enum.ex:1627: Enum.reduce/3
       test/ecto_crash_test.exs:13: (test)

Expected behavior
It should not crash!
Any advice? Thank you in advance!

Hi @walkr. For Error 1 and Error 2 it looks like the socket is closed (tcp recv: einval probably occurs due to a race condition when the socket is closed at approximately the same time that a :gen_tcp function is called, if the call is made earlier or later it is :closed). Assuming there isn't an issue with TCP there should be another log because a transaction/standalone query checkout from pool or sandbox ownership has exceeded the :timeout or :ownership_timeout. Assuming this is correct you may want to investigate why the time outs are occurring, or if you need to increase the relevant timeout.
For Error 3 the Repo's pool is being shutdown by its parent process, the Repo. The stacktrace is misleading/incorrect because of a bug in poolboy (incorrect use of :erlang.raise/3). Not sure why the Repo is shutting down without seeing log messages.
So I think you will need to read the logger output here to investigate further. If there isn't anything logged it may not be possible to offer any advise without seeing the code.

@walkr the bug is probably here:
https://github.com/walkr/ecto-crash/blob/master/lib/ecto_crash/server.ex#L40
That should be {:noreply, state}. Otherwise it is causing your worker to crash, which then causes the supervision tree to eventually restart or shutdown completely, which brings the postgrex connections down.

@josevalim When I am running the code that error never shows up in the logs.

Yes, that is indeed causing the error. I can't believe I've missed that. Thank you @josevalim!
However, crashing (2) below should not crash (1). Shouldn't their failures be handled independently, since I've specified the :one_for_one restart strategy?
Supervision tree
EctoCrash.Supervisor
    ├── EctoCrash.Repo   (1) ├── EctoCrash.Repo.Pool ├── ...
    └── EctoCrash.Server (2)

Code in question
  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    # Define workers and child supervisors to be supervised
    children = [
      supervisor(EctoCrash.Repo, []),
      worker(EctoCrash.Server, [:ok, []]),
    ]
    opts = [strategy: :one_for_one, name: EctoCrash.Supervisor]
    Supervisor.start_link(children, opts)
  end

And thank you for your reply as well @fishcakez

UPDATE: I've looked into it. It looks like the children do get restarted independently. It's only when running tests that the application shutdown is triggered. My mistake!
Thank you guys!

@walkr I was too lazy to notice there was linked code :blush:. You are correct that children get restarted independently but once max_restarts occur in max_seconds the supervisor reaches its max restart intensity and shuts down. The supervisor is the top supervisor in the application so the application stops.
@ericmj the timeout value isn't checked by :gen_server and a receive will be attempted without a try. This leads to the :gen_server code raising a :timeout_value error that isn't caught - so no GenServer log message. However a (sasl) :crash_report should be logged by :proc_lib, and ofc the supervisor reports.