Merge point clouds- data loss

Hi,
| Required Info                         |

Camera Model
{  D435 }

Firmware Version
(<IP_ADDRESS>)

Operating System & Version
Linux (Ubuntu 16.04)

Platform
PC

SDK Version
{ legacy / 2.. }

Language
{python }

Segment
{3d reconstruction}

Issue Description
I am using 4 realsense cameras for 3d reconstruction of object. I used export_ply_expmple.py code for creating 4 ply file. All cameras are covering all the required data(there is no any missing data). When I capture a rectangular shape object all the data is captured by cameras but when I captured a triangular shape object data is missing. I am getting a small gap between the merge of the two cameras ply file as below.

Will camera configuration and setting help to capture better ply file? When I observed the rgb and depth frame all data is visible.
Kindly help me out.

@Pratiksha2692 , can you be more specific and mark the  "problematic" are in the screenshot.
Also please elaborate how are the devices arranged, what is the exact shape of the "triangular" object and where is that object located with regards to each of the camera - a drawing/diagram/snapshot would really be helpful.

Thanks a lot for the reply.
I will send the drawing of the camera position in 1-2 days.

Hi,
Actually the attached screenshot is the human foot. It is the back view of foot. I am getting the missing data at the back of foot because of some unknown triangular geometry of it. But whereas for circular/ rectangular box no data is missing.

I had made the rectangular setup like 70cmX70cm.  4 cameras at 4 corner at some height and angle 35 degree. AOI is 45cmX45cm. All cameras are able to capture all the data.
Kindly advise.

@Pratiksha2692  Could you please provide the original snapshot before the merge? And what's the details about "angle 35 degree"? If you can provide your setup snapshot, that will be more helpful to understand. Thanks!

@Pratiksha2692 Did you get it through? Could you please update if you still need support? Thanks!

@Pratiksha2692 Any update from your side? Please provide more information if you still need support on this. Thanks!

Resolved


Autocorrection for Performance/ConstantRegexp is not working with pattern matching

Expected behavior
The autocorrection for Performance/ConstantRegexp should fix the offense even with pattern matching.
Actual behavior
The autocorrection for Performance/ConstantRegexp prepends the end delimiter /o, leading to the following error:
NODE_IN: unknown node (NODE_ONCE)

Steps to reproduce the problem
I have a method that uses pattern matching:
# constants/errors.rb
UNINITIALIZED_CONSTANT = "uninitialized constant".freeze
UNDEFINED_VARIABLE = "undefined local variable".freeze

def call
  case exception.message
  in /#{Constants::Errors::UNINITIALIZED_CONSTANT}/
    # do something
  in /#{Constants::Errors::UNDEFINED_VARIABLE}/
    # do something
  else
    # do something
  end
end

It works well but rubocop-performance raises the following offense:
Performance/ConstantRegexp: Extract this regexp into a constant, memoize it, or append an `/o` option to its options

The autocorrect prepends /o
# constants/errors.rb
UNINITIALIZED_CONSTANT = "uninitialized constant".freeze
UNDEFINED_VARIABLE = "undefined local variable".freeze

def call
  case exception.message
  in /#{Constants::Errors::UNINITIALIZED_CONSTANT}/o
    # do something
  in /#{Constants::Errors::UNDEFINED_VARIABLE}/o
    # do something
  else
    # do something
  end
end

and it breaks the code with the following error:
NODE_IN: unknown node (NODE_ONCE)

FYI I tried an if/else version and it works correctly
def call
  if exception.message.match?(/#{Constants::Errors::UNINITIALIZED_CONSTANT}/o)
    # do something
  elsif exception.message.match?(/#{Constants::Errors::UNDEFINED_VARIABLE}/o)
    # do something
  else
    # do something
  end
end

RuboCop version
rubocop (1.57.2)
rubocop-performance (1.19.1)

Note: I use the standard gem

Can you provide an executable code and steps to reproduce the NODE_IN: unknown node (NODE_ONCE) error?

Yes sure:
1/  Run the code below ruby script.rb
class Script
  # constants/errors.rb
  UNINITIALIZED_CONSTANT = "uninitialized constant".freeze
  UNDEFINED_VARIABLE = "undefined local variable".freeze

  class Error
    def self.call(exception)
      case exception
      in /#{UNINITIALIZED_CONSTANT}/
        p "UNINITIALIZED_CONSTANT"
      in /#{UNDEFINED_VARIABLE}/
        p "UNDEFINED_VARIABLE"
      else
        # noop
      end
    end
  end

  Error.call(UNINITIALIZED_CONSTANT)
end

2/ Run rubocop (as mentioned I use standard)
output:
script.rb:9:10: C: [Correctable] Performance/ConstantRegexp: Extract this regexp into a constant, memoize it, or append an /o option to its options.
      in /#{UNINITIALIZED_CONSTANT}/
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
script.rb:11:10: C: [Correctable] Performance/ConstantRegexp: Extract this regexp into a constant, memoize it, or append an /o option to its options.
      in /#{UNDEFINED_VARIABLE}/

3/ Run the autocorrect command
output:
script.rb: script.rb:5: NODE_IN: unknown node (NODE_ONCE) (SyntaxError)

Thanks.


Suggestion for small wiki update.

Section : Drawing shapes with IMDraw
instead of circle.Push(0) -> circle.Push(pixel.ZV).
During learning by doing, i realized circle.Push(0) didn't work so i replaced it with V(0,0).

Fixed! Thanks for catching this!


Use React.Fragment with multiple children instead of 'div'

Since peer dependency includes react 16, can make use of React.Fragment when multiple children are passed instead of wrapping it in another 'div'

Couple of issues with this:

We already return a fragment when there are multiple children present (an array of elements, compatible with all versions of react not just 16)

With one child we just return that element

The only case where we return a div is if you specify DOM attributes (style, className, etc.) on the MediaQuery component, which this would break since React.Fragment doesn't propagate those

So this PR doesn't add any new functionality, but breaks an existing feature.

You can see the logic here: https://github.com/contra/react-responsive/pull/168/files#diff-1fdf421c05c1140f6d71444ea2b27638R114

@contra ah thank you for pointing that out.  i just realized our project was using an older version of react-responsive which was getting an extra div wrapper and caused me to think it was coming from this.


Prepare Key Vault for beta release for November 2022.

API change check
API changes are not detected in this pull request.


Dialyzer error with using Sentry.plug in web routers

Environment

Elixir version (elixir -v): Elixir 1.8.2
Erlang/OTP version (erl): Erlang/OTP 22
Sentry version (mix deps): sentry 7.0
Operating system: macOS Mojave

Description
I've integrated Sentry with my elixir app, part of it is using sentry plug in our web router:
defmodule StudentsWeb.Router do
  use Plug.ErrorHandler
  use Sentry.Plug
  ....
end

After running mix dialyzer I got an invalid spec error:
lib/students_web/router.ex:3: Invalid type specification for function 'Elixir.StudentsWeb.Router':handle_errors/2. The success typing is
          (_, #{'reason' := _, _ => _}) ->
             'error' | 'excluded' | 'ignored' | 'nil' | 'unsampled' |
             {'ok',
              binary() |
              pid() |
              #{'__struct__' := 'Elixir.Task',
                'owner' := 'nil' | pid(),
                'pid' := 'nil' | pid(),
                'ref' := 'nil' | reference()}}

Looking into the plugin's codebase, it looks like related to the function declaration: https://github.com/getsentry/sentry-elixir/blob/master/lib/sentry/plug.ex#L125
Expected Behavior
Dialyzer works fine
Actual Behavior
Dialyzer throws an error about Invalid type specification for function :handle_errors
Steps to Reproduce

Run mix dialyzer on an elixir app after including Sentry.plug in a router module.

@islamhamdi thanks for opening an issue!
I was unable to reproduce with the example repository here: https://github.com/mitchellhenke/sentry_dialyzer_test
Is there something else going on in your router that could cause the issue?


Inconsistent cluster state

I'm trying to a full governance UI run through on devnet and running into some weird state issues. I'm using a mint from devnet to create a Treasury account, but the request for the AccountInfo returns null because it is connecting to the mainnet RPC url. Yet the cluster query string param is set to devnet and you can see the devnet toggle is also set.

after toggling devnet on and off again it seems to connect to the right cluster


Fix RenameKeywords incorrectly renaming the keywords with dots/underscores followed by space

Fixes #344

Codecov Report

Merging #351 (9a6ab0a) into main (b10aab9) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #351   +/-   ##
=======================================
  Coverage   96.72%   96.72%
=======================================
  Files          43       43
  Lines        3415     3416    +1
=======================================
+ Hits         3303     3304    +1
  Misses        112      112

Impacted Files
Coverage Δ

robotidy/transformers/RenameKeywords.py
97.72% <100.00%> (+0.05%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b10aab9...9a6ab0a. Read the comment docs.


Program fails if the [patch.crates-io] section must be considered in order to successfully resolve dependencies

Consider the following Cargo.toml.
[package]
name = "test-patch"
version = "0.1.0"
edition = "2021"

[dependencies]
wgpu = "0.18"
egui_wgpu_backend = "0.28"

[patch.crates-io]
wgpu = { git = "https://github.com/Imberflur/wgpu.git", tag = "0.18-with-fixes-for-veloren-v1" }

cargo check works fine here. But running cargo patch gives us this error:
Error: failed to select a version for `web-sys`.
    ... required by package `wgpu v0.18.0`
    ... which satisfies dependency `wgpu = "^0.18"` of package `test-patch v0.1.0 (/home/youser/src/test-patch)`
versions that meet the requirements `^0.3.64` (locked to 0.3.69) are: 0.3.69

the package `wgpu` depends on `web-sys`, with features: `GpuComputePassTimestampWrite` but `web-sys` does not have these features.

failed to select a version for `web-sys` which could resolve this conflict

We also get this error if we remove the [patch.crates-io] section from the Cargo.toml. The patch applied to wgpu is critical in this case, in order for cargo to resolve dependencies, and in consequence, in order for it to do anything at all.
I see that in the resolve_ws function, on lines 171-180, you have:
        let resolve: Resolve = resolve_with_previous(
            &mut registry,
            ws,
            &CliFeatures::new_all(true),
            HasDevUnits::No,
            prev.as_ref(),
            None,
            &[],
            false,
        )?;

That last false is the problem. Of course, if you just change it to true, that'll make the program fail on every project where a given target/patch/{PACKAGE} directory pointed to in the [patch.crates-io] section doesn't exist. Which is the situation you're probably in if you've freshly cloned a repository, or you ran cargo clean in a repository, that uses this program.
A direct solution would be to change the false to true, but (somehow) filter [patch.*] entries with a path pointing to a directory in target/patch out of the workspace being passed into this function. That would work in this case, but not if the patch itself is needed to resolve dependencies correctly (like for example, using cargo patch to apply the patches that make the wgpu fork work to wgpu). The only way to make that work would be to simply not resolve dependencies and just go grab the dependency from Cargo.toml directly, in which case the only way to know for sure which package to pick from crates.io would be to require pinned exact versions (e.g. wgpu = "=0.18.0" instead of wgpu = "0.18"), since you don't know if some transitive dependency actually requires a patch version that's lower than the highest one available. This, of course, doesn't apply to git repositories, which you can just clone (and switch to the branch or tag or commit if specified).

A note: in this case, the egui_wgpu_backend's wgpu version is different than the specified wgpu version, so you wouldn't want to do this anyway, but the core problem still exists and happens in real-world situations. Good enough for a test case.


Bump metadata schema to 0.4.1

This is what will change the official schema version in production. This should only be merged once we are ready for the meditor to use 0.4.1.
This should be merged simultaneously with dandi/dandi-api#323

@dchiquito - if this is done, the GUI will use this schema and not be able to use the meditor which will get the old schema metadata from the api.

I converted this to a draft PR just to avoid accidental merging.

Deprecated by #79


feature: dashboard interface widget could use relase/renew icon for dhcp interfaces

See title

I like this.

I was trying to work on this, thinking:
(1) Where would be possibly put this in the confined space of the widget.
(2) We only release, not renew. This may lead to fatal clicking in the dashboard section. Ideally the dashboard would be read-only. Don't know if there is anything that works against this rule that's in there?

Having one icon that pops up a menu with both choises? That solved both tie space and the accidental click issues

The confirm does work indeed. I also like the simplicity of splitting release / renew so we really can't offer both choices at the same time.
Lastly, there's the issue of redirecting. This might need an API call to work seamlessly, right now the release button redirects to the actual status page that does this. Needs more tinkering in my head.
Thanks for the feedback. :)

Out of scope: split of all common actions like add firewall rule, set interface properties,... Into a rest/json api ;)

That's a huge rework that needs proper sponsoring. We do what we can keeping the balance of "doesn't help to build it now on top of old code" and "it's new let's build this flexible from the start". We could end up reworking the code for a year practically get nowhere. :/
I'll keep you posted on this.

I said it was out of scope ;)

I tried to do this but it doesn't work very well on the dashboard as you could release your lease and end up with a disconnect by accident. I'm closing this.


[Bug]: create-config failing on start

Issue description
When running sugar create-config it directly aborts with the following error message:
sugar create-config
[1/2] 🍬 Sugar interactive config maker

Check out our Candy Machine config docs to learn about the options:
  -> https://docs.metaplex.com/candy-machine-v2/configuration

🛑 Error running command (re-run needed): invalid type: map, expected a sequence at line 7 column 18

sugar upload is resulting in the same error.
content of sugar.log attached.
Relevant log output
{"v":0,"name":"sugar","msg":"Lend me some sugar, I am your neighbor.","level":30,"hostname":"DESKTOP-97CGC42","pid":16973,"time":"2022-05-11T16:04:09.96045496Z","target":"sugar","line":101,"file":"src/main.rs"}

Priority this issue should have
Low (slightly annoying)

I am unable to repro, seeing if anyone else can rn

The problem reported was caused by a malformed metadata file, since the create-config tries to load the 0.json file if present. We need to improve the error reporting for deserialization errors - at a minimum, report which file generated the problem.


SudaWrite failure based on cwd?

Was struggling to debug this issue for a while, not sure if this is expected behaviour?
SudaWrite /etc/environment from / ->

SudoWrite /etc/environment from /etc -> success

Not sure but it seems https://github.com/lambdalisue/suda.vim/blob/master/autoload/suda.vim#L74 fails somehow...
What happen if you directly call suda#write() like (note that it will overwrite the content of /etc/foobar so BE CAREFUL!)
:call suda#write('suda:///etc/foobar')

And
:echo substitute('suda:///etc/foobar', '\v^(suda://)+', '', '')

:call suda#write('suda:///etc/foobar')

No errors, working as expected

:echo substitute('suda:///etc/foobar', '\v^(suda://)+', '', '')

/etc/foobar

Debugging:
plugin/suda.vim

args in line 28: "/etc/environment"

autoload/suda.vim

a:expr: <afile>
path in line 74 and onwards: "uda:///etc/environment"

It's quite strange. What happen if you modify the code like
function! suda#write(expr, ...) abort range
  echomsg "expr:" . a:expr
  echomsg "expand:" . expand(a:expr)
  echomsg "path:" . s:strip_prefix(expand(a:expr))
  " Original code starts from here
  let path = s:strip_prefix(expand(a:expr))
  " ...

Sometimes I have issues, sometimes not..
Successful (call nvim /etc/environment from user home directory):

Fail (call nvim /etc/environment from root directory):

What about
function! suda#write(expr, ...) abort range
  echomsg "expr:" . a:expr
  echomsg "expand:" . expand(a:expr)
  echomsg "bufname:" . bufname("%")
  echomsg "expand:" . expand("%")
  echomsg "path:" . s:strip_prefix(expand(a:expr))
  " Original code starts from here
  let path = s:strip_prefix(expand(a:expr))
  " ...

It's quite puzzling... I'm sorry but I've no idea...

Thought this was an issue with my Linux permissions after removing some shada files, but it seems that I can also reproduce this on my Mac. Just wondering if you are able to reproduce this as well?

How to reproduce the issue?

Fail (call nvim /etc/environment from root directory)

Then I could not reproduce it

@axieax I can not reproduce this either. Is your Nvim config available online?

Hi, yes it's available in my dotconfig repo: https://www.github.com/axieax/dotconfig

Have you tried reproducing this since your migration to lazy.nvim?


Create pool of test namespaces to enable cross-namespace Scenarios

Currently, we're creating a fixed cilium-test Namespace that contains a single stack of client/echo Pods. In order to support cross-namespace tests, we should move away from this model in order to make implementing new cross-namespace Scenarios less painful.
I propose to:

Make testing Namespaces randomly-generated
Create a sync.Pool (or similar) construct that allocates Namespaces on-demand (cattle, not pets)
Scenarios request one or more Namespaces at the start. If there are Namespaces in the pool that were Put() by finished Scenarios, they are re-used.
Make the harness take care of applying CNPs/KNPs to all Namespaces automatically, and to mutate the policies in a predictable way, like overwriting the Namespace field or overwriting certain annotations/labels. We could opt for placeholder values or (my preference) empty values (e.g. requiring metadata.namespace or certain label values or matchers to be empty).

This would also imply test Namespaces get removed when the connectivity test suite finishes. Remove cilium-test deletion behaviour from cilium uninstall as well.
Also, this brings us a step closer to being able to schedule Tests across Namespaces and being able to run them concurrently.

Any news on this? We have to delete the cilium-test namespace after the test ... a little sad when the test is part of some automation actually verifying the key core services of X Kubernetes cluster are running.
Thank you for the great product all in all - Cilium really kicks some tires.


help: is there a way to hide console/cmd ?

as title
I know it's very helpful show console/cmd for debug, but i want to hide it when release.

Probably default should be release, while for debugging, users can add debug argument in CLI. Just a suggestion.

This build option is part of build.zig, it is decided by the user, it should not be provided by the package, and as an imported package, it is theoretically impossible to modify the settings of the package that imports it.

But we can put it in the zig-webui readme as tips.


default.cow not found when running now/node tutorial

Going through the set up steps here: https://zeit.co/docs/v2/deployments/official-builders/node-js-now-node#when-to-use-it
But getting the following from my now deployment logs and a 502 from the running instance:
https://temp-imxkpb08t.now.sh/api/cowsay/?text=aoeu
Error: ENOENT: no such file or directory, open '/var/task/user/api/cows/default.cow'
    at Object.fs.openSync (fs.js:646:18)
    at Object.fs.readFileSync (fs.js:551:33)
    at Object.get (/var/task/user/api/cowsay/index.js:1:3111)
    at doIt (/var/task/user/api/cowsay/index.js:1:3949)
    at say$1 (/var/task/user/api/cowsay/index.js:1:3753)
    at Server.cowsay_1 (/var/task/user/api/cowsay/index.js:1:4247)
    at emitTwo (events.js:126:13)
    at Server.emit (events.js:214:7)
    at parserOnIncoming (_http_server.js:619:12)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:115:23)

Expected to see the cow output.

@timothyis

Hi @codeallthethingz and @liudonghua123! Would mind trying the most update to this example? It should have fixed this issue! https://zeit.co/docs/v2/deployments/official-builders/node-js-now-node 🙏

@timothyis, Thanks, it's works now.
BTW, I think the documentation is some misleading especially the file structure to the new one, could I make a pr to this documentation (https://zeit.co/docs/v2/deployments/official-builders/node-js-now-node ).

However, what's the difference of const cowsay = require("cowsay/build/cowsay.umd.js").say; and const cowsay = require("cowsay").say;, I tried locally, find nothing difference.

Here (https://github.com/piuccio/cowsay#usage-as-a-module) is the cowsay documentation which use var cowsay = require("cowsay");

Thanks for the speedy turn around @timothyis !


Fix libmagic issue with Python 3

For some reason libmagic return bytes instead of a string in Python 3, which can't be serialised to json

Coverage remained the same at 56.89% when pulling 0dd9fba16a73954a3bbb18c5b2de9995c07ef56f on r-darwish:filetypes-python3 into e09b3139eecd6aeca0c6acf48035ca3ffa96f87d on randomchars:master.


Check name with @Factory

At least, add a new test.
From: https://youtrack.jetbrains.com/issue/IDEA-157827

@juherr - Can you please let me know what test you are looking for? I will try and add tests based on what you need to get this ticket closed. Please let me know.

Not an issue, workaround found:
@Test(testName = "test")
public class GitHub1082Sample implements ITest {

    private final boolean value;
    private String testName;

    public GitHub1082Sample(boolean value) {
        this.value = value;
    }

    @Factory
    public static Object[] dp() {
        return new Object[]{
                new GitHub1082Sample(false),
                new GitHub1082Sample(true)
        };
    }

    @BeforeMethod
    public void setUp(Method method) {
        testName = method.getName();
    }

    public void testA() {
    }

    public void testB() {
    }

    @Override
    public String getTestName() {
        return testName + "(" + value + ")";
    }


[Hosts] Save with UTF8 BOM encoding

Summary of the Pull Request
I am not able to reproduce the issue described in the issue on my system.
PS C:\Users\david> [System.Text.Encoding]::Default.EncodingName
Unicode (UTF-8)

Without specify the encoding the file should be saved with the default encoding but I guess that saving it with UTF8-BOM should be fine.
Should this be a setting?

PR Checklist

[x] Closes: #23895
[ ] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Dev docs: Added/updated
[ ] New binaries: Added on the required places

[ ] JSON for signing for new binaries
[ ] WXS for installer for new binaries and localization folder
[ ] YML for CI pipeline for new test projects
[ ] YML for signed pipeline

[ ] Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

Change hosts file from the editor
Validate that the file is saved with UTF8 BOM encoding

Can we keep encoding of the existing file. Then we don't need a setting.

Detect encoding of a file could be challenging especially distinguish UTF8/UTF8-BOM. Wondering if we rellay need this extra complexity.
I have probably jumped to a solution without delving into the problem.
A feedback from the core team would be appreciated.

What is the default encoding of the file if it is installed with windows? I think we should us that.

I have checked on a few recently created VMs and the encoding is UTF8 without BOM but I guess it depends on OS default encoding.
By default the file should be saved witht the OS default encoding.

Hi @davidegiacometti ,
Thanks for opening this PR and contributing the fix.
Sorry for the delay in reviewing this PR.
From the looks of it, looks like a very specific use case, and the user is not even responding in the issue.
At this point, I think this makes mores sense as a setting disabled by default, since it might break something on other systems.
Does this make sense for you?

I totally agree!
Converted to draft as we only had one report.
Let's wait a response from the author before introducing the new setting.

That makes sense. Thanks a lot for helping here. :)

Closing due to lack of feedbacks.


upgrade to support dbt-core v1.2.0

We've just published the release cut of dbt-core 1.2.0, dbt-core 1.2.0rc1 (PyPI | GitHub release notes).
https://github.com/dbt-labs/dbt-core/discussions/5468 is an open discussion with more detailed information, and https://github.com/dbt-labs/dbt-core/issues/5474 is for keeping track of the communities progress on releasing 1.2.0
Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

[x] migrate necessary cross-db macros into adapter and ensure they're tested accordingly
[ ] remove any copy-and-pasted materialization (if your adapter inherits from another adapter)
[x] add new basic tests BaseDocsGenerate and BaseDocsGenReferences
[x] consider checking and testing support for Python 3.10

https://github.com/dbt-labs/dbt-core/pull/5432 might make it into the second release cut in the next week, in which case, you'll also might want to:

[x] implement method and tests for connection retry logic

Closing as a result of releasing 1.2.0 (https://github.com/cloudera/dbt-impala/releases/tag/v1.2.0)


Enhancement/Reduce bundle size

This pull request reduces the webpack bundle size by:

Splitting mixin.js into two modules so that the dependencies of each mixin only get included in the bundles of views that use the mixin and not in other bundles.
Removing all unused highlight.js languages.
Scoping the vue-chartjs imports.

The size savings resulting from these changes can be seen in the table below:

Before
After
Bundle

16K
16K
dataset

140K
140K
demo_named_entity

140K
140K
demo_text_classification

136K
136K
demo_translation

744K
176K
document_classification

728K
120K
download_seq2seq

728K
120K
download_sequence_labeling

728K
120K
download_text_classification

128K
128K
guideline

120K
120K
index

116K
116K
label

112K
112K
projects

740K
172K
seq2seq

744K
180K
sequence_labeling

520K
487K
stats

728K
120K
upload_seq2seq

728K
120K
upload_sequence_labeling

728K
120K
upload_text_classification

I built a tool to help you analyze webpack bundles for size regressions, and report them directly to GitHub PRs. It's free for open source, so it might be worth checking out and helpful to keep these optimizations from needing to happen again!
https://packtracker.io

Nice! Thanks for the pointer.
@Hironsan would you consider integrating @jondavidjohn's tool?


[Question]: Reproduction of Big Bench Hard with LLMLingua-2

Describe the issue
As there is no example for compression of BBH prompts given in the experiments folder I wanted to ask, which parameters were used for the compression of the CoT prompts used for big bench hard? Specifically, which values for the params

compression_rate / target_token
force_tokens
force_reserve_digit

were used, and was use_context_level_filter set to true?
Furthermore, there are 3 CoT example prompts for each task, how was the compression done, were they passed as a single string or as a list, with each CoT prompt as a separate item, as done for the GSM8K CoT prompts here?

Hi @pzs19, could you help to answer this question?

Thank you!

@pzs19 Sorry, I do have a follow-up question regarding your response:

1. "target_token" is described in "Tokens" column of the Table 3 in our [paper](https://aclanthology.org/2024.findings-acl.57.pdf).

I was of the impression that the column "Tokens" was the actual achieved token count, not the target_token parameter? Otherwise it should be the same value for 1-shot constraint and half-shot constraint, respectively, but in reality its a different value for each compression method?

The meaning of "1-/half-shot constraint" is here

Thank you!


Onlyoffice behind Apache reverse proxy and owncloud

Hi Community,
I installed onlyoffice on a Server behind a apache revese proxy.
due to the server having multiple other websites behind it including an owncloud I can't put onlyoffice on /
I  put it ot /onlyoffice but the onlyoffice server alway redirects it to /welcome , which I added in the proxy but then it tells me : "Something went wrong during installation"
I tried to apply the apache configuration from the onlyoffice website but it didn't work.
When I entered the website I got  hostname , hostename/welcome
Is ther a configration file or spot in the database where I can change the sitename?

You can more details with an example config for Apache to describe the problem? In Docker we used as a proxy server with nginx https://github.com/ONLYOFFICE/docker-onlyoffice-owncloud/blob/master/nginx.conf.

Hi @r4a5a88,
You can see how to install documentserver in virtual path here.

hi @agolybev
I used that configuration on my server.
It didn't work

thats the apache conf for the onlyoffice site and a test-owncloud i am using to test it
Alias testcloud /testcloud
Alias onlyoffice /onlyoffice

#SetEnvIf Host "^(.*)$" THE_HOST=$1
#RequestHeader setifempty X-Forwarded-Proto https
#RequestHeader setifempty X-Forwarded-Host %{THE_HOST}e
#ProxyAddHeaders Off

ProxyPass /testcloud http://<IP_ADDRESS>/owncloud
    	ProxyPassReverse /testcloud http://<IP_ADDRESS>/owncloud

ProxyPassMatch (.*)(\/websocket)$ "ws://<IP_ADDRESS>/onlyoffice/$1$2"
ProxyPass /onlyoffice "http://<IP_ADDRESS>/"
ProxyPassReverse /onlyoffice "http://<IP_ADDRESS>"
ProxyPass	/welcome	http://<IP_ADDRESS>/welcome
ProxyPassReverse	/welcome http://<IP_ADDRESS>/welcome

@r4a5a88
Fisrt I'm not sure for
Alias testcloud /testcloud
Alias onlyoffice /onlyoffice

Because you use this key word below in the config
Next,  this statements are required:
  SetEnvIf Host "^(.*)$" THE_HOST=$1
  RequestHeader setifempty X-Forwarded-Proto http
  RequestHeader setifempty X-Forwarded-Host %{THE_HOST}e
  RequestHeader edit X-Forwarded-Host (.*) $1${VPATH}
  ProxyAddHeaders Off

Finaly, I think you  don't have to proxy welcome path by this:
ProxyPass	/welcome	http://<IP_ADDRESS>/welcome
ProxyPassReverse	/welcome http://<IP_ADDRESS>/welcome

Try using this config:
Listen 80
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so

<IfModule unixd_module>
  User daemon
  Group daemon
</IfModule>

Define VPATH /onlyoffice
Define DS_ADDRESS <IP_ADDRESS>

<Location ${VPATH}>
  Require all granted
  SetEnvIf Host "^(.*)$" THE_HOST=$1
  RequestHeader setifempty X-Forwarded-Proto http
  RequestHeader setifempty X-Forwarded-Host %{THE_HOST}e
  RequestHeader edit X-Forwarded-Host (.*) $1${VPATH}
  ProxyAddHeaders Off
</Location>

ProxyPassMatch ^\${VPATH}(.*)(\/websocket)$ "ws://${DS_ADDRESS}/$1$2"
ProxyPass ${VPATH} "http://${DS_ADDRESS}/"
ProxyPassReverse ${VPATH} "http://${DS_ADDRESS}/"

It won't proxy to owncloud, but you can test proxy to the documentserver.

thanks for the conf.
It works. I get a green screen.
One more problem. In my cloud , when I open a document , I only see a white screen.

Hi @r4a5a88.
Yes this config is suitable only for HTTP traffic. Try changing string RequestHeader setifempty X-Forwarded-Proto http to RequestHeader setifempty X-Forwarded-Proto https

Vielen Herzlichen Dank @agolybev
Es funktioniert :)


removed grammatical errors

NOTE: Please read the CONTRIBUTING.md guidelines before submitting your patch,
and ensure you followed them all:
https://github.com/Kong/kong/blob/master/CONTRIBUTING.md#contributing
Summary
SUMMARY_GOES_HERE
Full changelog

Removed all errors

Issues resolved
Fix #4937

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Sorry, but this adds more mistakes than it resolves.


Load Error when jekyll tries to install generate-projects.rb

I got the following message when jekyll tried to install the generate-projects.rb (I'm running the build in Travis CI):
home/travis/build/jputrino/f5-openstack-docs/_plugins/generate_projects.rb:64:in `require': cannot load such file -- zip/zip (LoadError)
    from /home/travis/build/jputrino/f5-openstack-docs/_plugins/generate_projects.rb:64:in `<top (required)>'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:74:in `require'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:74:in `block (2 levels) in require_plugin_files'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:73:in `each'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:73:in `block in require_plugin_files'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:72:in `each'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:72:in `require_plugin_files'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/plugin_manager.rb:18:in `conscientious_require'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/site.rb:79:in `setup'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/site.rb:41:in `initialize'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/commands/build.rb:29:in `new'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/commands/build.rb:29:in `process'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `call'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `block in execute'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `each'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `execute'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary/program.rb:42:in `go'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/mercenary-0.3.5/lib/mercenary.rb:19:in `program'
    from /home/travis/.rvm/gems/ruby-2.2.0/gems/jekyll-2.5.3/bin/jekyll:20:in `<top (required)>'
    from /home/travis/.rvm/gems/ruby-2.2.0/bin/jekyll:23:in `load'
    from /home/travis/.rvm/gems/ruby-2.2.0/bin/jekyll:23:in `<main>'
    from /home/travis/.rvm/gems/ruby-2.2.0/bin/ruby_executable_hooks:15:in `eval'
    from /home/travis/.rvm/gems/ruby-2.2.0/bin/ruby_executable_hooks:15:in `<main>'

The command "./script/cibuild.sh" exited with 1.

I also ran into this problem and gem install zip didn't really cut it for me. Because Rubyzip was updated with a major release, there is no backwards compatibility (they removed the Zip prefix in class names, which you can see in this plugin). See this note at the rubyzip repo.

Apologies for the (huge) delay here. This project is no longer maintained, and I'm archiving it.


format certain files in the root directory

This adds an additional glob to the default formatter to include some files in the root directory. Which files to target is a bit tricky. Generally speaking, Gro expects everything to be in your src/ directory, but it's common to have some files in the root directory for various tools, and we want a subset of those to be formatted. We don't want to format things like package-lock.json or package.json because those are modified directly by npm. Insead of adding a blacklist, I opted to just target TypeScript, JavaScript, and Markdown files. Seems ok?

Is it possible there might be other ts, js or md files someone other than us might keep in their root that they don't want formatted? Honestly a code cleaner targeting files outside the src directory feels like unexpected behavior to me. This seems like a potential sharp edge for users, especially considering it targets a different subset of files than it does in the src folder.

Good point, I can see how that's surprising behavior. In the Node ecosystem it's common to have *.config.js (or ts) files in the root directory, along with the normal README, CHANGELOG, CONTRIBUTING, etc. I personally want those formatted and can't think of any I wouldn't, but it's weird that it omits all other file types. I'm unable to think of examples I wouldn't want formatted but the special casing makes it look wrong.
I noticed this before but just confirmed - prettier and npm are formatting package.json and package-lock.json identically. So at least for our projects with current behavior, we could use the same set of file extensions. If CI ever breaks because of npm not matching the formatting, we could re-evaluate. How does that sound? gro format works consistently on both the root directory and src/?

Alright cool, that works for me.

Great alright it now uses the same set of extensions.


Preview window background changing

Current Preview window background is not optimal.
I don't know why you prefer (240,240,240) grey color, but for example search results list uses classic white (255,255,255) color, but preview window still uses grey... And considering that very big amount of images has white backgroung, they looks a bit worse now then they may be on the white background.
A simple example:

There is a bit but important differense between these backgrounds.  I bet that users would prefer white bg because of it.
Current color:

More suitable background:

Maybe someone prefers another colors...

So I suggest to make an option to change Preview window background color in options.
...or at least make it white.

I don't "prefer" light grey, that's just the default background color. And the problem with setting it white by default is that themes won't work anymore (ex: #432).
I guess an option to change it could be added in the settings though, but wouldn't that duplicate issue #813 in some way?

I don't "prefer" light grey, that's just the default background color. And the problem with setting it white by default is that themes won't work anymore (ex: #432).
I guess an option to change it could be added in the settings though, but wouldn't that duplicate issue #813 in some way?

Both #432 and #813 are centered in changing all GUI colors, so, they talks about THEMES or skins, or darkmode, anyway they talks about changing all colors.
My suggestion is only about images background in preview window, even not in all window, but at least in image-area. And not becuase of I prefer another colors or somethimg, but because of almost 50% of all images have white bg, and better is not to see edges as I shown in top of this topic. Many other image viewers have an option to set background color for images(FastStone Viewer for example), without any other color customisation, because in this case It is  really important.


[$500] Workspace - #admin room with messages shows skeleton loading after workspace is deleted

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.56-0
Reproducible in staging?: y
Reproducible in production?: n
Issue reported by: Applause - Internal Team
Action Performed:

Go to staging.new.expensify.com.
Create a new workspace.
Go to #admin room.
Send a message.
Delete the workspace.
Go to #admin room.

Expected Result:
#admin room will not show skeleton loading after the workspace is deleted.
Actual Result:
#admin room shows skeleton loading after the workspace is deleted.
Workaround:
n/a
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/7435ef39-86ee-4cf3-9698-2f59dddf58d0
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0190baa2ce2890e3e7
Upwork Job ID:<PHONE_NUMBER>906227712
Last Price Increase: 2024-03-25

We think that this bug might be related to #vip-vsb

The GetOlderActions command is erroring out, sample logs: 86894ae17a5e2d72-YVR

Seems like the functions loadOlderChats and loadNewerChats gets called in a infinite loop:

Adding external and bug to see if we can get a contributor to help here

@aldo-expensify , does this need to be a deploy blocker? I have a feeling it might be related to comment linking, multiple bugs have been reported, like here https://expensify.slack.com/archives/C049HHMV9SM/p1711147499521539
cc @roryabraham , you likely have a better idea if it's related.

@aldo-expensify , does this need to be a deploy blocker?

I think so, it is causing a regression that doesn't allow opening the admin room.

Proposal
Please re-state the problem that we are trying to solve in this issue.
The app keeps calling the loading skeleton after a workspace is deleted
What is the root cause of that problem?
the reportActions returned from the getContinuousReportActionChain always have fewer actions after deleting the workspace
this happens because the optimistic value of the delete workspace action doesn't have a previousReportActionID so the conditions here fails after one iteration

What changes do you think we should make in order to solve the problem?
we can modify the endIndex loop so that it skips if the current action is close
    while (
        (endIndex < sortedReportActions.length - 1 && sortedReportActions[endIndex].previousReportActionID === sortedReportActions[endIndex + 1].reportActionID) ||
        !!sortedReportActions[endIndex + 1]?.whisperedToAccountIDs?.length ||
        !!sortedReportActions[endIndex]?.whisperedToAccountIDs?.length ||
        sortedReportActions[endIndex]?.actionName === CONST.REPORT.ACTIONS.TYPE.ROOMCHANGELOG.INVITE_TO_ROOM ||
        sortedReportActions[endIndex]?.actionName === CONST.REPORT.ACTIONS.TYPE.CLOSED ||
        sortedReportActions[endIndex + 1]?.actionName === CONST.REPORT.ACTIONS.TYPE.CLOSED ||
        sortedReportActions[endIndex + 1]?.actionName === CONST.REPORT.ACTIONS.TYPE.CREATED
    ) {

POC:
https://github.com/Expensify/App/assets/59809993/bbbfc15f-9d78-4102-9190-49100ed15d86
alternatively:
or we can pass a previousReportActionID to the buildOptimisticClosedReportAction and set it to the last action id so that the first condition passes for the optimistic action.

Thanks for the proposal @abzokhattab
There are two Closed action which looks like the reason for the issue. Why do we have two Closed Actions in the first place? One of them don't have previousReportActionID which probably is the one optimistically added.

This PR  might have caused the regression.

Does reverting the PR and adding previousReportActionID optimistically solves both this issue and https://github.com/Expensify/App/issues/35174 issue?

I reverted the PR and yes indeed the current issue is not there anymore, however, the task and the admin room will not be set as archived in offline mode and I can still send new messages in the deleted rooms:
https://github.com/Expensify/App/assets/59809993/0fef1b1a-b888-4058-88a0-9a5b00d99874
Reverting the pr makes the reports list as empty [] thus it will not entered into this loop and the optimistic values will not be added:
https://github.com/Expensify/App/blob/d3ada5bfc09dde810411c46f71bf93caf4156fef/src/libs/actions/Policy.ts#L331-L368
I think that we still need to optimistic action to prevent this issue

I guess it's also a backend issue. When we close the report, we do not reuse reportAction from optimistic data. Check it https://github.com/Expensify/App/issues/35952
Also https://github.com/Expensify/App/pull/30269/files#diff-cd0dfd9f8af5bc7cd33ff4f0337ef70574d42424ceecd7692b193348d3d8ecd4R869-R876

I guess it's also a backend issue. When we close the report, we do not reuse reportAction from optimistic data

That makes sense, the problem is gone if we log out and log back in.

Looking at the DeleteWorkspace payload, we are not sending any client side reportActionID.

hmm, but sending an optimistic reportActionID for the CLOSED action is not really possible I think because there could be thousands of reports getting closed because of the workspace deletion.

Shouldn't we remove the optimistic CLOSED  reportAction using the success data? I see that in DeleteWorkspace response we have the real CLOSED reportActions coming.

I created a PR here https://github.com/Expensify/App/pull/39008 that removes the optimistic CLOSED action on success and it seems to fix the problem:
https://github.com/Expensify/App/assets/87341702/8c95b790-64f1-4dc4-b9f1-e682371fdc1f
I'm not sure if it could cause another problem or if it is fine. I'm also not sure if we should do the same on failure too.

@roryabraham @perunt when you have some time, can you check if this solution makes sense to you? 🙏

PR is being CP'd presently

This hit production two days ago 🎉
Gonna close this, doesn't look like @sobitneupane reviewed the PR.  If I'm wrong, please comment, reopen.

https://github.com/Expensify/Expensify/issues/383435

@mallenexpensify : @ishpaul777 ended up reviewing

ohh thanks for clearing that up!


[MINOR] Fix typos in hudi-utilities module

Change Logs
As above.
Impact
Only typo fixes.
Risk level
none
Documentation Update
N/A
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

CI is green.


F# .Net Standard 2.0 + Entity Framework Core

I am trying to use EF Core on F# but this error appears
FS1109	A reference to the type 'Microsoft.EntityFrameworkCore.Infrastructure.WarningsConfigurationBuilder' in assembly 'Microsoft.EntityFrameworkCore' was found, but the type could not be found in that assembly

Steps to reproduce
Added NuGet package
Microsoft.EntityFrameworkCore    2.0.1
MySql.Data.EntityFrameworkCore     8.0.9-dmr

module Contexto

open Microsoft.EntityFrameworkCore

type BloggingContext() =
    inherit DbContext()
        member val Blogs: DbSet<Blogs> = Unchecked.defaultof<DbSet<library.MyClass>> with get, set    // (*"valor padrao")
         //public DbSet<Post> Posts { get; set; }

        override OnConfiguring(optionsBuilder: DbContextOptionsBuilder) =
            optionsBuilder.UseMySQL(@"Server=<IP_ADDRESS>;Database=myDataBase;Uid=root;Pwd=abcf;")
            |> ignore

Further technical details
EF Core version: 2.0.1
Database Provider: MySql.Data.EntityFrameworkCore 8.0.9-dmr
Operating system: Windows 10 Pro 64bits
IDE: Visual Studio 2017 15.6.0 Preview 1.0

I downgraded MySql.Data.EntityFrameworkCore to version 6.10.5
then the Project compiles OK.

If I try to use this F# Project containing this Entity Framework Core, from a WPF Project, this error appears:
Unknown build error, 'Could not load type 'Microsoft.EntityFrameworkCore.Infrastructure.DesignTimeProviderServicesAttribute' from assembly 'Microsoft.EntityFrameworkCore, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60'.'	WpfApp2

Could you share your project? It seems like there may be some missing binding redirects or incompatible NuGet packages...

Sure. Can you please ask somebody to do a blog posting about:
Basic example just like the C# version ( Blog / Postings ) from https://docs.microsoft.com/en-us/ef/core/
How to use Entity Framework Core in F#
My buggy solution, in a ZIP file
https://drive.google.com/open?id=1FLBCzAsuk49SriiA9a216ieebcOyFBPR

I suspect a lot of your pain is coming from referencing a "new" F# project (.NET Standard class library) from a "classic" one (WPF). I know the C# and VB projects required a bit of work in this area to make it a nice experience...

The MySQL provider may also not be compatible with EF Core 2.x yet.

This F# + .Net Standard + .Net Framework + Nuget packages
seems to have some integration problems.
I also tried to use F# SqlProvider

FS3053	O provedor de tipos 'FSharp.Data.Sql.SqlTypeProvider' relatou um erro: O construtor do provedor de tipos gerou uma exceção: Invalid host of cross-targeting type provider. Exception: System.NullReferenceException: Referência de objeto não definida para uma instância de um objeto.   em <StartupCode$FSharp-Data-SqlProvider>.$ProvidedTypes.referencedAssemblyPaths@8509.GenerateNext(IEnumerable1& next)   em Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.MoveNextImpl()   em Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase1.System-Collections-IEnumerator-MoveNext()   em Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable1 source)   em ProviderImplementation.ProvidedTypes.ProvidedTypesContext.Create(TypeProviderConfig cfg, FSharpOption`1 assemblyReplacementMap)

More exceptions:

System.TypeLoadException: 'Method ApplyServices on type MySql.Data.EntityFrameworkCore.Infraestructure.MySQLOptionsExtension do assembly MySql.Data.EntityFrameworkCore, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=c5687fc88969c44d does not have an implementation.'

I am looking for a solution where I can use C#, F#, ASP.Net, ASP.Net Core, WPF, UWP, Xamarin, Entity Framework (Core), MongoDB (CosmosDB) seamlessy. I think that Microsoft should do a blog posting with all this. I am a little confused.

@TonyHenrique Curious as to why you want to use all those technologies in a single solution. Can you provide some details?

I just tried to mix up everything avaliable in the .Net
This solution is just a test I am doing.
The team here is using PHP, MySQL, C# (.Net Core)
Then I went to test if

F# + the mentioned databases / connection ways would work well.
Interop between the .Net Core and .Net 4.7.1
Xamarin Forms
WPF
UWP

etc.
I could make it work / interop well using .Net 4.7.1, Entity Framework 6
When it comes to .Net Core, EF Core, I could not make it work / interop well.
The problem is on .Net 4.7.1  I will not be able to use these Class Libraries on Xamarin Forms.

Hey @TonyHenrique, if you reference your .NET Standard library from a .NET Core console app, do you still see the same issue as when you reference it from your WPF project?

Just saw this line in the release notes for Visual Studio 2017 version 15.6 Preview 3:

.NET Standard projects can now be referenced by F#/.NET Framework projects.

Yep, this is now enabled, though there is currently an issue in that because .NET Standard projects use the .NET Core SDK in VS, and the way in which FSharp.Core is resolved in that context is different than with .NET Framework. Thus, it's likely that doing this will result in a runtime error where the assembly manifests for FSharp.Core are mismatched.
Long story short is that today, it's probably in one's best interest to reference .NET Standard components from .NET Core components just so that tooling issues don't get in the way.

Visual Studio 2017 version 15.6 includes several improvements to F# tooling's .NET Standard support (see release notes for details).
I am going to go ahead and close this issue as external, because there doesn't seem to be any actionable item for EF Core and I hope the underlying issue has been resolved.
If it turns out that this still repros with Visual Studio 2017 15.6, please consider creating a new issue at https://github.com/Microsoft/visualfsharp/issues/new.


FactoriesFinder should be able to ignore files without extension.

I have come across this issue, when using SVN (sigh, I know) FactoriesFinder throws an error related to missing extension for entries file. This fixes the issue.

+1 I'm also using SVN and getting the same error, this fixes it.


Add Dockerfile security scanning to all images

We should ensure that our check Dockerfiles are built with best practices in mind.  Now that we have moved our builds to github actions, this should be easy enough to implement.
This would hopefully identify the issue brought up in #400 and cause us to fix the problem.

Fixed in #406


Form Submission on Enter

Use Keyboard to navigate options and press enter immediately Form is get Submitted

Possible solutions are avoiding Enter keypress but i need enter key press too.

I have the same needs. This stackoverflow post helped me: https://stackoverflow.com/questions/11388251/google-autocomplete-enter-to-select.

This should be an open issue because it's easy to fix if you are inside the Google directive.  I'd argue really this should be fixed inside the underlying Google component because it'll happen for them as well outside of angular.  But this is what I did to work around it.  Inside the component holding the form element I looked for the enter key bubbling up.
    @ViewChild( 'street', { read: ElementRef} ) streetElem : ElementRef;

    @HostListener('keydown', ['$event'])
    handleKeyboardEvent( event : KeyboardEvent ) {
        // this prevents places component from submitting the form when using enter
        let key = event.key.toLowerCase();
        if( key == 'enter' && event.target === this.streetElem.nativeElement ) {
            event.preventDefault();
            event.stopPropagation();
        }
    }

@skynet2 With Chrome's built in autocomplete the code submitted in 2.0.3 will produce an error when the user clicks on the address suggested by the browser.

This can be fixed by wrapping the code in a if( "key" in event ){}
I believe the code snippet below should resolve this issue.
@HostListener('keydown', ['$event'])
	handleKeyboardEvent( event : KeyboardEvent ) {
		// this prevents places component from submitting the form when using enter
		if( "key" in event ){
			let key = event.key.toLowerCase();
			if( key == 'enter' && event.target === this.streetElem.nativeElement ) {
				event.preventDefault();
				event.stopPropagation();
			}
		}
	}


Allow using Active Directory groups in global distribution groups

Describe the solution you'd like
AD groups can only be added to an app distribution group. When I try to add an AD group to a global distribution group I don't see them.
This causes quite a lot more setup for us. We have different groups we try to maps for each of our apps: QA, iOS dev, Android dev, Team A, Team B...
We have all these entities mapped in our Active Directory.. It would have been easier to setup distribution groups once for our AppCenter organisation and link apps to each of these groups.
Describe alternatives you've considered
Right now I have to manually go to each app and recreate Distribution groups mapped to AD groups. We have other 15 apps, this is a lot of additionnal setup.

Thanks for the feature request @lisional - I can definitely understand how the alternative is tedious. @botatoes or @blparr may be able to help out here.

Hi @lisional , thanks a lot for your feature request. This is definitely something in our radar. I don't have an ETA I can share with you just yet, but I'll keep this feature request open to make sure we update you when we do.
Thanks!
Blanca

Hi @lisional , thanks a lot for your feature request. This is definitely something in our radar. I don't have an ETA I can share with you just yet, but I'll keep this feature request open to make sure we update you when we do.
Thanks!
Blanca

Can You please update about this feature ETA.
Thanks !
Priyanka

We had no update on this issue from the team, but I discovered today that it seems to have been implemented.
I was able to create global distribution group and add an Active Directory group as a tester


JSDoc-like for cuid vs uuid

Problem

model One {
  id String @id @default(uuid())
}

model Two {
  id String @id @default(cuid())
}

In the above as a user I don't know what uuid/cuid are exactly, why both are there, which to use, when.
I don't want to navigate to the Prisma docs because it will take too much time.
I don't want to open my browser and google because it will take too much time.
Suggested solution

Prisma should incorporate summaries of the knowledge existing about them into JSDoc like info, so that I can hover in my IDE to get most of the info I could need, plus examples, with links therein for complete info from some spec, article, etc. on the web.

Alternatives

Additional context

CC @sorenbs


Change meta data service timeout to 200ms

As per spec

API change check
API changes are not detected in this pull request.


Make the pages pop

🍿

I wonder if you could make the issue more clear!!

Do you want different page links on the website to open up in new tabs ?!

I'm sorry this was more of a joke 👅 Closing this :)


⚠ Bofrost.fr has degraded performance

In a080383, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 997 ms

Resolved: Bofrost.fr performance has improved in 623c268 after 6 minutes.


readiness/liveness probes

We've been testing upgrades with other pods and noticed that upgrades went more smoothly when we added readiness/liveness probes to our deployments.
We don't see readiness/liveness probes for the SparkApplication CRD.  Is there anything we need to add to our SparkApplication definition to ensure minimal or no downtime if we have to update the image we're using with our Spark Application?

What kind of readiness/liveness checks do you have in mind? They are generally useful for long-running serving workloads, and less for batch workloads IMO.

we have long running Streaming Spark Applications that could be affected if we had to switch out the Spark Operator while the Streaming application was running.

@jdelgadillo could you describe what failed during your tests?

@skonto We don't have failing tests.  We're just planning for upgrades.  We've found that having readiness/liveness probes helps reduce or eliminate downtime for our deployments.  As we were looking into how we'd upgrade the Spark operator, we were wondering if there was a way to configure the probes for our Spark applications--specifically ones that are executing long running streaming jobs.

Btw I have a failure but not sure if liveness could help: https://issues.apache.org/jira/browse/SPARK-27900

An alternative question: What are the best practices for upgrading the Google Spark Operator and Spark Applications?

@jdelgadillo the operator can generally be safely upgraded while having running SparkApplictions as long as the new version is based on a compatible version of Spark. For example, if both the new and old versions of the operator are based on the same minor version of Spark, e.g., 2.4.x. While the operator is offline, the status section of your SparkApplictions won't be updated. But things will catch up once the operator is back online. We maintain API backward compatibility in all beta versions. There might be behavior changes between versions that are not backward compatible. But in this case, we will make sure we provide an upgrade guide in the documentation.

I also have similar requirement and need liveness probe to be added for spark application. @jdelgadillo did you find any solution for this ?

@infa-angoel we wound up using monitoring via prometheus/grafana to ensure our pods were running as expected


responsive-img not responsive in small screen, IE11 and FF40.0.3

Hello!
My first round with Materialize, why responsive-img class works with Large and Middle screens but stop working with small screens. Only Opera and Chrome works correctly. Something I'm missing?
Best regards.

Hi @pedroserapio, could you provide a codepen?

Hello @MulletPr00f , thank you for the reply.
Have a look, please.
http://codepen.io/Serapio/full/WQQXvr/
I hope you can see that if the image is placed outside of a card, stop being responsive when the screen is small.
My best regards.

Because you didn't define the small grid size and only m and l were defined


Remember the selected sidebar section

Probably need to think about how macOS deals with multiple windows in this scenario.


Trumbowyg doesn't update ngModel

I am using Trumbowyg in my Angular2
I try to update my object property [(ngModel)]='scene.description'
trumbowyg doesn't work
<trumbowyg
[(initialContent)]="scene.description" [options]="options1"
[(ngModel)]="scene.description"
(savedContent)="scene.description=$event">

How I can update my model from Trumbowyg ?

Did you check that: https://github.com/monad98/ng2-lazy-trumbowyg ?
I do not use Angular, and I did not know what's your code behind your trumbowyg component, so I cannot help you as is.


Why archive in git tree vs being a release?

ok, I'll take a release, and put files in release note, thanks for your suggest.


Add updateTexture & some shortcuts in Sprite; Cosmetic

Add updateText.  If a sprite is created by a canvas ,  when the canvas's changed , user could call updateText for updating sprite.
Add some shortcuts for accessing  some properties of sprite._texture.
Cosmetic Text. Just define some local vars for accessing properties of this , for DRY.

Agree on "this" in Text, but updateTexture.. hm. Its handy for single sprites that are created from canvases, but I dont know if its good to put it there.
@GoodBoyDigital @bigtimebuddy what about "SpriteCanvas" class, that is created from canvas and not from texture? Or with canvas. Move part of Text there, add this updateTexture method.

How about these shortcuts ?

And If use SpriteCanvas ,  do we need  TextureCanvas & BaseTextureCanvas too ???

@finscn definitely no. Only for sprite.

I like the local variable caching, that should help minifying output. Small, but good change.
I disagree with adding baseTexture, orig, frame, and trim getters for Sprite. It's bloats the API and doesn't provide a lot of convenience. These things are already publicly available sprite.texture.baseTexture, sprite.texture.orig, etc.
Also, I don't understand the need to add updateTexture. This functionality is also already available:
sprite.texture.baseTexture.update() (example: https://jsfiddle.net/bedrdc1b/).
So in summary, I like the local variable caching, but disagree with the API changes.

ping @ivanpopelyshev


Read the Docs build fails with Poetry >= 1.8.0

Read the Docs build fails with Poetry >= 1.8.0.
Read the Docs builds 23584649.
python -m sphinx -T -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
Running Sphinx v7.2.6
Traceback (most recent call last):
⋮
  File "/home/docs/checkouts/readthedocs.org/user_builds/sethfischer-osr/checkouts/102/docs/conf.py", line 6, in <module>
    import sphinx_rtd_theme  # type: ignore[import-untyped] # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'sphinx_rtd_theme'

See python-poetry/poetry#9025.

Fixed with 00380b17c0d9fd09a020e755429fafcdcc174589.


Technologies cannot be researched due to hidden prerequisite

Mod source
Factorio Mod Portal
Which mod are you having an issue with?

[ ] pyalienlife
[ ] pyalternativeenergy
[ ] pycoalprocessing
[ ] pyfusionenergy
[ ] pyhightech
[X] pyindustry
[ ] pypetroleumhandling
[ ] pypostprocessing
[ ] pyrawores

Operating system

=Windows 10

What kind of issue is this?

[ ] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[X] Progression
[ ] Balance
[ ] Pypostprocessing failure
[ ] Other

What is the problem?
Technology "production-science-pack" depends on "railway", which is hidden.
Mod List

Py Industry

Steps to reproduce

Research all prerequisites
Tech will still be red

Workaround: Executing the following will allow the tech to be researched.
/c game.player.force.technologies["railway"].researched = true

Additional context
This was reported on the Bob's mods forum. As Py Industry already includes some compatibility changes for Bob's Logistics, it makes sense for the issue to be fixed there. Also that the issue exists without any of Bob's mods.
Bob has mostly retired from modding now. I've taken over maintenance of Bob's mods.
Edit: Decided to fix the Bob's Logistics issue on Bob's side. Removed references to technologies added by Bob's from the issue report. Just left the base game Production Science Pack issue.

Closing this, thanks for the fix.


Too many AudioContext created

Error in event handler for runtime.onMessage: NotSupportedError: Failed to construct 'AudioContext': The number of hardware contexts provided (6) is greater than or equal to the maximum bound (6).

Since the content.js is no longer injected multiple time, this could be moved to a global variable...

Is this also somehow related to #8?


Failed to start the daemon: Failed to start dqlite server

Required information

Distribution: Ubuntu
Distribution version: Focal
The output of "lxc info" or if that fails:

Kernel version: 5.4.0-72-generic
LXC version: lxc (1:4.0.2-0ubuntu1) [universe]
(Comes from stable Focal Cloudinit image which pulls from Universe)
LXD version: lxd (1:0.9) [universe]
(Comes from stable Focal Cloudinit image which pulls from Universe)
Storage backend in use: zfs

Issue description
Solutions-qa hit this on a run over the weekend during an Openstack deploy. So the lxd/lxc Snaps come from the stable Focal Cloudinit image we are using. I would be willing to bet we hit a database transaction corruption due to unclean system shutdown or something similar. I came across canonical/raft#57 and noticed you could not figure out exactly what happened so I figured I'd throw a bug together in case we have any valuable info for you in our logging.
Our logging is linked below. The environment has unfortunately since been cleaned up. If you have any questions let me know!
Feel free to move this to Raft if that is a more appropriate location.
Here's the usual failure snippet from syslog:
Apr 18 16:28:25 spearow systemd[1]: Started Service for snap application lxd.daemon.
Apr 18 16:28:25 spearow lxd.daemon[5506]: => Preparing the system (20037)
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Loading snap configuration
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Setting up mntns symlink (mnt:[4026533887])
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Setting up kmod wrapper
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Preparing /boot
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Preparing a clean copy of /run
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Preparing /run/bin
Apr 18 16:28:25 spearow lxd.daemon[5506]: ==> Preparing a clean copy of /etc
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Preparing a clean copy of /usr/share/misc
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Setting up ceph configuration
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Setting up LVM configuration
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Rotating logs
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Setting up ZFS (0.8)
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Escaping the systemd cgroups
Apr 18 16:28:26 spearow lxd.daemon[5506]: ====> Detected cgroup V1
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Escaping the systemd process resource limits
Apr 18 16:28:26 spearow lxd.daemon[5506]: ==> Disabling shiftfs on this kernel (auto)
Apr 18 16:28:26 spearow lxd.daemon[5506]: => Re-using existing LXCFS
Apr 18 16:28:26 spearow lxd.daemon[5506]: => Starting LXD
Apr 18 16:28:26 spearow lxd.daemon[5643]: t=2021-04-18T16:28:26+0000 lvl=warn msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
Apr 18 16:28:26 spearow lxd.daemon[5643]: t=2021-04-18T16:28:26+0000 lvl=warn msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
Apr 18 16:28:26 spearow lxd.daemon[5643]: t=2021-04-18T16:28:26+0000 lvl=eror msg="Failed to start the daemon: Failed to start dqlite server: raft_start(): io: load closed segment<PHONE_NUMBER>000001-0000000000000001: file is empty"
Apr 18 16:28:26 spearow lxd.daemon[5643]: Error: Failed to start dqlite server: raft_start(): io: load closed segment<PHONE_NUMBER>000001-0000000000000001: file is empty
Apr 18 16:28:27 spearow lxd.daemon[5506]: => LXD failed to start
Apr 18 16:28:27 spearow systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Apr 18 16:28:27 spearow systemd[1]: snap.lxd.daemon.service: Failed with result 'exit-code'.
Apr 18 16:28:27 spearow systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 1.

Information to attach
Test run on our dashboard:
https://solutions.qa.canonical.com/testruns/testRun/8d916282-adc3-4a2b-b977-d77aeb6b642c
Full Artifacts/Logs/Bundles:
https://oil-jenkins.canonical.com/artifacts/8d916282-adc3-4a2b-b977-d77aeb6b642c/index.html
Juju model crashdump (syslogs/etc from Machine 3 that encountered this issue)
https://oil-jenkins.canonical.com/artifacts/8d916282-adc3-4a2b-b977-d77aeb6b642c/generated/generated/openstack/juju-crashdump-openstack-2021-04-18-20.17.41.tar.gz

Looks like we don't collect /var/snap/lxd. And since the environment has been cleaned and redeployed I can't get to it.
I'll add that to crashdump collection for the future though.

Thanks, without it, there's really nothing else we can do, especially since we've fixed a number of bugs in that area over the past weeks so this may have been resolved already.


Optimize locking behavior of the profiling handler.

The profiling handler is in the path of our most performance critical components (especially the activator). Taking a write-lock on each request is probably a bad idea.
Replaced the mutex with an atomic flag. Lost reads are not critical in this code path and that should be the quickest solution in terms of avoiding contention.
/assign @vagababov

/assign @mattmoor
This seems to need superpowers.


Switch from distutils Version to packaging.version to avoid deprecation warning

Hi,
for some time, my build were trashed with deprecation warning like this:
/something/.venv/lib/python3.9/site-packages/pytest_sugar.py:169: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

I'd love to get rid of them and looks like this little patch can do the trick. This package already depends on packaging module so should be easy to merge.
Please, let me know if I can help you in any way to get this merged soon.
Best regards,
Tomasz.

I examined the repo a little and it looks like it's not updated for past years. Travis tests not working well, etc. I tried to migrate from travis to github actions, so at least you will know if PR pass tests.
Consider it a proof of concept, If you will be interested, we can try to polish it to final version.

I'm having the same problem and my console is being trashed for the time being. Do we have an ETA on a merge?

Thank you 🥳

@Teemu can you release a new version with this fix, please?


feat: Allow passing arguments to test script (issue #88)

As per issue #88
Background: Recently the npm repo cucumber-nightwatch was deprecated in favour of nightwatch-api, whose example for how to use CucumberJs with Nightwatch involves using start-server-and-test. What the example does not account for that cucumber-nightwatch supported though was passing arguments to Cucumber for tags, targeting feature files, etc.
This limitation was due to the passing-arguments feature not yet implemented in start-server-and-test. I'm currently working on a project where we need to update nightwatch to a version that the old cucumber-nightwatch does not support, so I needed something that works and took on this issue.
Their example: https://github.com/mucsi96/nightwatch-api/tree/master/examples/cucumber
I opened an issue on their end as well: https://github.com/mucsi96/nightwatch-api/issues/24
I've tried my best to copy the style of the repo, hopefully it is up to par!
Usage:
Assuming:
{
    "test": "start-server-and-test test:setup http-get://localhost:4444 test:run",
    "test:run": "my-test-runner",
    "test:setup": "my-server-setup",
}

npm run test -- --tag mytag

would pass -- --tag mytag to the test:run script, which would run my-test-runner --tag mytag.
I've kept in the convention of -- as a requirement for this to run correctly as it seems to be how passing arguments to other scripts is typically handled. There's other ways to do this as well, but this is what better matched what we had previously.
Thanks for this repo!

it would be so so nice to have an "end-to-end" test script to confirm that this is working as expected

@rgeraldporter That was very useful. May I know when this PR will be merged so that I can update start-server-and-test dependency in my nightwatch-api framework and allow it to accept arguments

@rgeraldporter can you update the code to fix the npm run demo and other tests?
$ npm run demo
><EMAIL_ADDRESS>demo /home/travis/build/bahmutov/start-server-and-test
> node bin/start.js http://<IP_ADDRESS>:9000
starting server using command "npm run start"
and when url "http://<IP_ADDRESS>:9000" is responding
/home/travis/build/bahmutov/start-server-and-test/bin/start.js:36
console.log(`running tests using command "${test.join(' ')}"`)
                                                 ^
TypeError: test.join is not a function
    at Object.<anonymous> (/home/travis/build/bahmutov/start-server-and-test/bin/start.js:36:50)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)

in https://travis-ci.org/bahmutov/start-server-and-test/builds/432655532


TLS callhome establishment happens with v1.2 followed by v1 attempts

Dear Team,
We are using TLS call-home feature for client server connection establishment with netopeer2-server 1.1.77 & netopeer2-cli 2.0.64.
Here the client server establishments can happen only with tlsv1.2, lesser versions are omitted.
But what we see from tcpdump, client is sending hello using tlsv1 with multiple attempts & finally it tries to connect using tlsv1.2. Due to which the connection establishment is taking more time.
Below are the server logs where the connection timeout happened for tlsv1.
Timed out after 5s (Operation now in progress).
Trying to connect the pending socket 213.
Timed out after 5s (Operation now in progress).
Failed to connect socket 213 after 5 retries, closing.
Trying to connect via IPv6 to fe80::205:b9ff:fe52:8500.
Trying to connect the pending socket 213.
Timed out after 5s (Operation now in progress).

For reference, tcpdump shows the retries with tlsv1:

Also please let us know why nearly 20 secs taken for tlsv1 handshake. What makes such delay? Incase of tlsv1.2, no such delays seen. For reference, screenshot of a tlsv1 handshake attached:

How do we make changes in netopeer2 client side, so that any attempts are only with tlsv1.2?

Openssl installed in our system is 1.0.2 - hope it supports TLSv1.2.
Got an information from google that Openssl versions till 1.0.0h supports SSLv2, SSLv3 and TLSv1.0. From Openssl 1.0.1 onward support for TLSv1.1 and TLSv1.2 is added.
In that case, probably highest supported version TLSv1.2 must be negotiated as you pointed out. But why is it trying initially with lower version 1.1 & then goes to higher version 1.2? Are you still recommending us to upgrade openssl? if yes, which version of openssl are you recommending?
:: OpenSSL Version in our system ::
[bs1122@siq5g-vm2 onecell5g]$ openssl
OpenSSL> version
OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL>
[bs1122@siq5g-vm2]$ openssl ciphers -v | awk '{print $2}' | sort | uniq
SSLv3
TLSv1.2

You are using an unsupported version, which is a security risk, additional to the compatibility problems. Use a supported version.


Create LICENSE

Pull Request Test Coverage Report for Build 4

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.429%

Totals

Change from base Build 2:
0.0%

Covered Lines:
27

Relevant Lines:
28

💛  - Coveralls

Pull Request Test Coverage Report for Build 4

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.429%

Totals

Change from base Build 2:
0.0%

Covered Lines:
27

Relevant Lines:
28

💛  - Coveralls

Pull Request Test Coverage Report for Build 4

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.429%

Totals

Change from base Build 2:
0.0%

Covered Lines:
27

Relevant Lines:
28

💛  - Coveralls


⚠ Samsung has degraded performance

In 918d340, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2762 ms

Resolved: Samsung performance has improved in 8488567.


Proper handling of runtime-exceptions in jitter and ping

Describe the bug
When the jitter or ping operation is performed on IP addresses which are not valid, it leads to flooding of logs in the terminal. This needs to be avoided and rather a single line indicating teh kind of failure should be shown.

Handling of runtime exceptions in a broader sense requires to know what all exceptions and errors can arise in the runtime of a process, say: jitter analysis. @Harkishen-Singh Please modify #115 with respect to this. Handling exceptions should be aimed at handling any mishaps that break the process in any way(for example: A faulty URI and how we handle that when one occurs, do we stop the current goroutine, or the entire thread). That was an implementation example, not a real case scenario. The issue mentioned here deals with more of logging/debugging rather than handling anything. So, we have to either rename the issue or extend it. Thoughts?

In case there is an invalid URL or when the CLI returns an error, we can create a separate module to handle all the errors, handling all the different conditions possible. Like two of the possible conditions can be, one is invalid URL and the other one checking the internet connection and accordingly, the log can be maintained with proper details and the issue. I think it's better to open a separate issue for this. @aquibbaig @Harkishen-Singh what do you think?

@muskankhedia yeah, that would be better. And accordingly, the user can be alerted by the alerting system (once its ready to)

With the merge of #288, this is covered. Hence, closing.


GitHub build & quality assurance workflow updates

This PR introduces the following changes to the GitHub build & QA workflows:

Fixed 32-bit Linux build

ssdeep & GeoIP are not available as 32-bit packages in the Ubuntu 22.04, so they're not installed. This also means that the build configurations without them are redundant.

Fixed clang builds on Linux (both on 64-bit & 32-bit)
Installed YAJL package on macOS builds as it's a required dependency.

Additionally, this fixed the execution of tests on those builds (previously, execution would show that 0 tests were run as YAJL is necessary to parse test files).

Include build configurations 'without libxml' to Linux/macOS/Windows builds

This required annotating regression testcases that depend on libxml support.
Updating the regression test executable to detect whether it was built with/without libxml support in order to run/skip those testcases.

Include build configuration 'with lmdb' to Linux/macOS builds.

There is currently a 'without lmdb' configuration in both platforms, but LMDB is disabled by default and only included in the build if explicitly turn on.

Removed 'without YAJL' builds from Linux/macOS build configurations as it's a required 3rd party dependency.
Added without ssdeep build to Linux build configurations.
Moved cppcheck (check-static) out of every Linux build configuration and run this checks only once to improve workflow times.
Updated actions/checkout version on Linux/macOS build to remove deprecation warnings on workflow execution.
Simplified and standardize job names.

Hi @eduar-hte,
thanks again this awesome PR!

This PR introduces the following changes to the GitHub build & QA workflows:
* Fixed 32-bit Linux build
  * ssdeep & GeoIP are not available as 32-bit packages in the Ubuntu 22.04, so they're not installed. This also means that the build configurations without them are redundant.

These are great, these were on my list as I realized that the 32 bit build is the same as the 64 bit. Thanks.

* Fixed clang builds on Linux (both on 64-bit & 32-bit)
* Installed YAJL package on macOS builds as it's a required dependency.
  * Additionally, this fixed the execution of tests on those builds (previously, execution would show that 0 tests were run as YAJL is necessary to parse test files).

oh, thanks!

* Include build configurations 'without libxml' to Linux/macOS/Windows builds
  * This required annotating regression testcases that depend on libxml support.
  * Updating the regression test executable to detect whether it was built with/without libxml support in order to run/skip those testcases.

Hmmm... I'm a bit confused here. Why did you add this? As we discussed (or I suggested) here XML and JSON parsers should be mandatory too. So I think we don't need --without-xml option - what do you think?

* Removed 'without YAJL' builds from Linux/macOS build configurations as it's a required 3rd party dependency.

Yes, this is a good step.

* Added without ssdeep build to Linux build configurations.

Thanks.

* Moved cppcheck (check-static) out of every Linux build configuration and run this checks only once to improve workflow times.

Ah, this is very good, thanks.

* Updated actions/checkout version on Linux/macOS build to remove deprecation warnings on workflow execution.
* Simplified and standardize job names.

Thanks for all.
Here you changed the enabled value from 1 to 0 - what is the reason that you disabled these tests?
Thanks again.

There is a new SonarCloud issue, but I think actually we can mark it as FP, so you don't need to care of that.

Here you changed the enabled value from 1 to 0 - what is the reason that you disabled these tests?

Oh, that is a mistake. Those are the tests that are disabled on Windows (because of the incorrect dependency on a specific order from iterating on a std::unordered_map container), which I should not have included in that commit.
I've just reverted those changes. Nice catch!

Include build configurations 'without libxml' to Linux/macOS/Windows builds

This required annotating regression testcases that depend on libxml support.
Updating the regression test executable to detect whether it was built with/without libxml support in order to run/skip those testcases.

Hmmm... I'm a bit confused here. Why did you add this? As we discussed (or I suggested) here XML and JSON parsers should be mandatory too. So I think we don't need --without-xml option - what do you think?

Yes, you mentioned in passing that it's an important component of the library. At the same time, in the limited scope of this PR (just to update the GH workflows) I thought that given that it's not a mandatory dependency and the codebase currently has support to disable use of that library (through the compiler definition WITH_LIBXML2), the QA runs should try and cover that configuration.
Of course, a separate PR could be done to remove those compiler guards and this new build configuration, and I'd be ok with it (it didn't take a lot of effort to add support for regression for that resource and annotate the testcases).

Yes, you mentioned in passing that it's an important component of the library. At the same time, in the limited scope of this PR (just to update the GH workflows) I thought that given that it's not a mandatory dependency and the codebase currently has support to disable use of that library (through the compiler definition WITH_LIBXML2), the QA runs should try and cover that configuration.

With regards to dependencies, I think the section in README.md could be updated to be more accurate. The line that mentions YAJL, libpcre & libXML2 starts by stating that these are mandatory dependencies but clarifies that two out of the three are not yet so. I think either pcre or pcre2 are mandatory to build the library, while as we discussed libXML2 seems optional (at least for the time). I think that sentence would be more accurate and read better like this:

Other dependencies include YAJL, as ModSecurity uses JSON for producing logs and its testing framework, libpcre  for processing regular expressions in SecRules, and libXML2 (not yet mandatory) which is used for parsing XML requests.

One other minor issue is that the last sentence implies that libinjection is not mandatory, and that if the library is missing you just won't get the associated operator, but something I found out a few times when I forgot to init & update submodules on my Unix builds is that configure won't complete without it, which actually makes it  mandatory. :-)
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: error:

  libInjection was not found within ModSecurity source directory.

  libInjection code is available as part of ModSecurity source code in a format
  of a git-submodule. git-submodule allow us to specify the correct version of
  libInjection and still uses the libInjection repository to download it.

  You can download libInjection using git:

     $ git submodule init
     $ git submodule update

I updated the Unix build configurations with the following change:

Include build configuration 'with lmdb' to Linux/macOS builds.

There is currently a 'without lmdb' configuration in both platforms, but LMDB is disabled by default and only included in the build if explicitly turn on.

Oh, that is a mistake. Those are the tests that are disabled on Windows (because of the incorrect dependency on a specific order from iterating on a std::unordered_map container), which I should not have included in that commit.
I've just reverted those changes. Nice catch!

no worries, thanks!

Yes, you mentioned in passing that it's an important component of the library. At the same time, in the limited scope of this PR (just to update the GH workflows) I thought that given that it's not a mandatory dependency and the codebase currently has support to disable use of that library (through the compiler definition WITH_LIBXML2), the QA runs should try and cover that configuration.
Of course, a separate PR could be done to remove those compiler guards and this new build configuration, and I'd be ok with it (it didn't take a lot of effort to add support for regression for that resource and annotate the testcases).

Okay, we can cover all possible build options, therefore if we enable the flag --without-xml then we also need to check the codebase with --without-yajl - what do you think?

BTW, because SonarCloud configuration is not done through a GitHub workflow I couldn't see if it was possible to configure it to analyze the code as standard C++ 17.

I try to take a look how is it possible.

It currently reports issues and suggests using newer features from C++20 (such as std::format or ranges, see here and here) which add a bit of noise.

Just FYI: there is an intention to bump the C++ version to 20 - see this comment.

Of course, a separate PR could be done to remove those compiler guards and this new build configuration, and I'd be ok with it (it didn't take a lot of effort to add support for regression for that resource and annotate the testcases).

Okay, we can cover all possible build options, therefore if we enable the flag --without-xml then we also need to check the codebase with --without-yajl - what do you think?

I thought about that but didn't do it because I wanted to follow what the project documentation currently states.
Moreover, the last couple of days I worked on what I suggested in the previous comment, about actually enforcing the mandatory dependencies (yajl, pcre & libxml2) to simplify the build (reduces the number of combinations to test across OS & architectures) and codebase (reduces unnecessary #ifdef blocks).
I'd like to like to follow up this PR with those changes, which I didn't want to include in this one in order to keep this one simple (and aligned with the current documentation), and then analyze and discuss those changes separately. What do you think?

With regards to dependencies, I think the section in README.md could be updated to be more accurate.

Agree. Thanks for spotting that.

The line that mentions YAJL, libpcre & libXML2 starts by stating that these are mandatory dependencies but clarifies that two out of the three are not yet so. I think either pcre or pcre2 are mandatory to build the library, while as we discussed libXML2 seems optional (at least for the time). I think that sentence would be more accurate and read better like this:
* Other dependencies include YAJL, as ModSecurity uses JSON for producing logs and its testing framework, libpcre  for processing regular expressions in SecRules, and libXML2 (not yet mandatory) which is used for parsing XML requests.

or we should make them really mandatory.

One other minor issue is that the last sentence implies that libinjection is not mandatory, and that if the library is missing you just won't get the associated operator, but something I found out a few times when I forgot to init & update submodules on my Unix builds is that configure won't complete without it, which actually makes it mandatory. :-)
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: error:

  libInjection was not found within ModSecurity source directory.

  libInjection code is available as part of ModSecurity source code in a format
  of a git-submodule. git-submodule allow us to specify the correct version of
  libInjection and still uses the libInjection repository to download it.

  You can download libInjection using git:

     $ git submodule init
     $ git submodule update

Yeah, unfortunately libinjection is an orphaned project (I mean there are not so much updates and maintaining), so operating systems does not like these libraries. So I think actually the best way is to use that as submodule. But yes, I know it s*cks. :)

(edit) I ended up creating another quick PR (#3145) to move this discussion there and update README, but I think it'd be better if I create a PR to actually update configure and the codebase to enforce the mandatory dependencies (such as YAJL, libXML2 and probably PCRE2 -as the original pcre has been end of life since 2021-).

Right, thank you.

I updated the Unix build configurations with the following change:
* Include build configuration 'with lmdb' to Linux/macOS builds.

  * There is currently a 'without lmdb' configuration in both platforms, but LMDB is disabled by default and only included in the build if explicitly turn on.

If you look at the configure step of builds (such as this), you'll see in all the configurations (not just for the one in the strategy created to build 'without lmdb'):
   + YAJL                                          ....found v2.1.0
      -lyajl, -DWITH_YAJL -I/usr/include/yajl
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.13
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2

Nice catch! Seems like this has changed meanwhile, because LMDB was included if the configure script detected it. But yes, this modification is necessary, thanks.

The line that mentions YAJL, libpcre & libXML2 starts by stating that these are mandatory dependencies but clarifies that two out of the three are not yet so. I think either pcre or pcre2 are mandatory to build the library, while as we discussed libXML2 seems optional (at least for the time). I think that sentence would be more accurate and read better like this:

or we should make them really mandatory.

I agree, that's why I started working on that separate PR I just mentioned to do that (and then update README.md 🙂).

Yeah, unfortunately libinjection is an orphaned project (I mean there are not so much updates and maintaining), so operating systems does not like these libraries. So I think actually the best way is to use that as submodule. But yes, I know it s*cks. :)

Curiously, while working on enforcing the mandatory dependencies I realized that configure actually states that both submodules are mandatory dependencies!
ModSecurity -  for Linux

 Mandatory dependencies
   + libInjection                                  ....
   + SecLang tests                                 ....a3d4405

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (MaxMind) v1.5.2

Okay, we can cover all possible build options, therefore if we enable the flag --without-xml then we also need to check the codebase with --without-yajl - what do you think?

I thought about that but didn't do it because I wanted to follow what the project documentation currently states.

oh, that makes sense - thanks for clarify it.

Moreover, the last couple of days I worked on what I suggested in the previous comment, about actually enforcing the mandatory dependencies (yajl, pcre & libxml2) to simplify the build (reduces the number of combinations to test across OS & architectures) and codebase (reduces unnecessary #ifdef blocks).
I'd like to like to follow up this PR with those changes, which I didn't want to include in this one in order to keep this one simple (and aligned with the current documentation), and then analyze and discuss those changes separately. What do you think?

Agree - I'll take a look again this PR and probably will merge it soon. Then you can send the next one.
Thank you for your patience and your work.

or we should make them really mandatory.

I agree, that's why I started working on that separate PR I just mentioned to do that (and then update README.md 🙂).

Right,

Curiously, while working on enforcing the mandatory dependencies I realized that configure actually states that both submodules are mandatory dependencies!
ModSecurity -  for Linux

 Mandatory dependencies
   + libInjection                                  ....
   + SecLang tests                                 ....a3d4405

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (MaxMind) v1.5.2

Hmm... I don't see the reason why "SecLang tests" is mandatory - probably we should figure out this. Libinjection is... well, I think it's like pcre(2), so yes, I can agree that.

Thank you again this great PR, merging it.

You're welcome. I'll follow up later about adjusting the mandatory dependencies after rebasing.


This commit was done to fix existing issue of validations.py file

Closes: #1
Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!


bluetti-mqtt HAOS add-on no longer working after latest upgrade

Core 2024.1.3 Supervisor 2023.12.0 Operating System 11.4 Frontend 20240104.0
I've been encountering this error since I upgrade HAOS to the latest version. I'm assuming this has to do with python 3.10 package? Since there was no update on bluetti_mqtt add-on since 05/2023.
Checking the directory /usr/local/lib/ there is nothing there except /perl5 and [which python3.10] gives me not found.
Traceback (most recent call last):   File "/usr/local/lib/python3.10/site-packages/bluetti_mqtt/bluetooth/client.py", line 78, in _connect     await self.client.connect()   File "/usr/local/lib/python3.10/site-packages/bleak/__init__.py", line 531, in connect     return await self._backend.connect(**kwargs)   File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/client.py", line 141, in connect     async with async_timeout(timeout):   File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__     self._do_exit(exc_type)   File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit     raise asyncio.TimeoutError asyncio.exceptions.TimeoutError 2024-01-15 10:06:38 ERROR    Error connecting to device 05:8F:0E:AD:2B:5D: Traceback (most recent call last):   File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/client.py", line 204, in connect     reply = await self._bus.call(   File "/usr/local/lib/python3.10/site-packages/dbus_fast/aio/message_bus.py", line 371, in call     await future asyncio.exceptions.CancelledError
Any help appreciated, thank you!
I've posted this in the add-on git as well. Wasn't sure which git would be able to resolve.
https://github.com/SSMCD/bluetti2mqtt

closed, looks like there was a connection issue lol.