Broken Initial vs. Optimized budget allocation plot and CSV data starting from v3.3.2

Project Robyn
Describe issue
We've found that starting from v3.3.2 charts of reallocated budget expense were broken.
I performed some investigation and found that before v3.3.2 SUM(initSpendShare) in *_reallocated.csv files 1.0
But starting from v3.3.2 it has strange values (see screenshot attached)
I was testing it on the same data with the same model configuration, just tried different versions of Robyn from GitHub.
I compared versions in git and found following change # e775ac2cd4d50daa5fd3f0c318fd4f744f0cf8a1
Rolling back the change in R/R/allocator.R fixes the bug.
Could you review if this change introduces the bug?
Provide dummy data & model configuration
I cannot provide real data for obvious reasons. But I can show comparison of generated PNG and CSV files:

Allocator execution code:
    AllocatorCollect <- robyn_allocator(
      InputCollect = InputCollect
      , OutputCollect = OutputCollect
      , select_model = "1_1_4"
      , scenario = "max_historical_response"
      , channel_constr_low = c(0.7, ..., 0.7, 0.01)
      , channel_constr_up = c(1.5, ..., 1.5, 1)
    )

IMO there is obvious bug in robyn_allocator() which leads to incorrect values of initSpendShare
Environment & Robyn version
Robyn MMM versions: v3.3.2 and above (tested up to v3.4.5 including #main branch)
R version: 4.1.1
Platform: Win10

hey thanks for spotting this! it's indeed a bug. The change was intended though, just with the wrong variable. The correction is here fca1d04dec69dc2be75941dc354482a223607438

I tested it few times. The plot looked Ok from first look, but there is something still be wrong with data in *_reallocated.csv files: the sum(initSpendShare) != 1
I got following sum value in several models: 0.966, 0.956, 0.966, 0.966 (rounded to 3 digits)

These couple of models have paid var with 0 coefficients, right? Then it is intended. For example, if OOH has coef 0, then it has no response and thus is excluded from the allocator. At the same time, the total spend will still include OOH spend. It means the OOH spend will be distributed to other channels. Can you confirm if it's the case?

I need a help:
Where can I look at coefficient for the paid vars? sorry, I'm lost in gammas, thetas and other params.
Is it something we set or something what's saved in the Robyn model (in RDS file)?
Another observation:
Right now sum(optmSpendUnit) == optmSpendUnitTotal == initSpendUnitTotal
But shouldn't it be sum(optmSpendUnit) == sum(histSpendUnit) ?
The sum(histSpendUnit) is less than initSpendUnitTotal.
That's why sum(initSpendShare) < 1.0 in my case.

No worries. All coefficients can be found either in the pareto_aggregated.csv or OutputCollect$xDecompAgg.
You're asking about hyperparameters (gammas, thetas etc.)? Users set a range and Robyn figures out which values within the ranges are best.
Using the simulated data, I've selected a model that has facebook_I coef is 0 and used it in allocator. Below is the screenshot of output. When I created this table, I've tried to name the columns as intuitive as possible. But apparently it needs more explanations, since some other users are also asking for column definition. Let me do it here and will include it into official documentation in the future.

$ channels                 : Equals to paid_media_vars. Those with coefficient = 0 are excluded in allocator because it doesn't make sense to optimise 0 response channel.
$ histSpend                : Total spend of each channel within modelling window.
$ histSpendTotal           : Sum of histSpend, including excluded channels. This also represents total historical marketing budget.
$ initSpendUnitTotal       : Sum of initSpendUnit, including excluded channels.
$ initSpendUnit            : Mean spend of each channel for the respective time grain. Note that the mean is calculated with non-zero spend period. For example, when a new channel has spend c(0, 0, 0, 5, 5), mean spend is 10/2 = 5, because we want mean spend to reflect active spending decision of marketing.
$ initSpendShare           : equals initSpendUnit / initSpendUnitTotal. Sum of initSpendShare is <1 when 0-coef channels are excluded.
$ initResponseUnit         : Response of initSpendUnit of each channel for the respective time grain.
$ initResponseUnitTotal    : Sum of initResponseUnit. Excluded channels don't impact it here because they have 0 response anyway.
$ initRoiUnit              : Equals  initResponseUnit / initSpendUnit
$ expSpendTotal            : Total marketing budget for optimal allocation. When scenario = "max_historical_response", it equals histSpendTotal. When scenario = "max_response_expected_spend", it equals to user's input in expected_spend argument in robyn_allocator() function
$ expSpendUnitTotal        : Total marketing budget for optimal allocation for the respective time grain. When scenario = "max_historical_response", it equals to initSpendUnitTotal. When scenario = "max_response_expected_spend", it equals to expected_spend / expected_days
$ expSpendUnitDelta        : Equals expSpendUnitTotal / initSpendUnitTotal -1
$ optmSpendUnit            : Optimised spend of each channel for the respective time grain.
$ optmSpendUnitDelta       : Equals optmSpendUnit / initSpendUnit -1. This also reflects the channel constraints in channel_constr_low and channel_constr_up.
$ optmSpendUnitTotal       : Equals sum(optmSpendUnit). When scenario = "max_historical_response", it equals to initSpendUnitTotal and expSpendUnitTotal. However when channel_constr_up is too restrictive, it might be lower that the latter two metrics. When scenario = "max_response_expected_spend", it equals to expected_spend / expected_days. However when channel_constr_up is too restrictive, it might be lower that the latter one.
$ optmSpendUnitTotalDelta  : Equals optmSpendUnitTotal / initSpendUnitTotal -1.
$ optmSpendShareUnit       : Equals optmSpendUnit / sum(optmSpendUnit)
$ optmResponseUnit         : Response level of optmSpendUnit for each channel
$ optmResponseUnitTotal    : Equals sum(optmResponseUnit)
$ optmRoiUnit              : Equals optmResponseUnit / optmSpendUnit
$ optmResponseUnitLift     : Equals optmResponseUnit / initResponseUnit - 1
$ optmResponseUnitTotalLift: Equals optmResponseUnitTotal / initResponseUnitTotal - 1

Finally, to answer your question: sum(initSpendUnit) < sum(optmSpendUnit) when there're channels with 0 coefficients and thus excluded from the budget allocation. For example, assuming you spend total budget of 100$ into TV=70$, OOH=20$ & Search=10$. When OOH has 0 coefficient/ has no response, sum(initSpendUnit) == 80$. But when doing allocation, you still want to spend 100$. So the 20$ of OOH will be split into remaining channels. Hope it makes sense.

Thank you for detailed explanation!


add encoding algorithm to RFC 0592

RFC 0592 doesn't include the encoding algorithm for indy credential attributes that is required for interoperability. RFC 0036 described this, but the v2 protocol doesn't. This adds the encoding algorithm to RFC 0592 and updates the AIP2 commit link + changelog.
Signed-off-by: Timo Glastra<EMAIL_ADDRESS>

Discussed WG 20220209. Plan to merge next meeting. Longer review process given the update to AIP2.0 as a clarification.

Merged after longer review cycle.


Token expiration

Before submitting a pr:

[x] Tests passing
[x] Black formatting
[x] Rebase/merge the dev branch
[x] Note in the CHANGELOG

What to do about the issued to username/consignee, also with regard to Anands open PR that relies on this funcitonality?
https://github.com/ScilifelabDataCentre/dds_cli/blob/1dee5d61104a9f3fe6c135a8fed1cd38d9442069/dds_cli/user.py#L322-L324

As I wrote in Anand's PR:
This can be handled by showing the username that the user has entered to login. There is no need to add this to the plaintext part of the token and extract it from there. So if cli can get the final token to use the endpoints, the entered username can be displayed as logged in user.


update library pointers

@sanuann - i think the demo protocol should be updated with the changes in the github workflow in this PR. so that we actually check that the protocol is valid.


[5.3] IRGen: Correctly compute the offset for a non-fixed field after…

… an empty field
Introduce getByteOffsetDuringLayout() so that we can treat empty fields
just like other fixed sized field while performing layout.
Explanation: We used to incorrectly compute the offset of a non-fixed
field after an empty field inside of a closure capture context. This lead to
crashes at runtime because we read memory that the wrong offset.
Scope: Causes Swift programs that caputre empty fields and non-fixed
fields to crash or misbehave.
Risk: Medium, we now correctly compute the offset in such cases.
Testing: Regression test added.
Reviewer: Slava Pestov
rdar://62349871

@swift-ci Please test

Original PR: https://github.com/apple/swift/pull/31863


volatility: metadata column names containing periods cannot be interpreted

Bug Description
If metadata column names contain periods, that column cannot be viewed on the volatility plot. It seems that the periods may be interpreted as special characters.
Steps to reproduce the behavior
see here (sourced from forum xref below)
Expected behavior
Those columns should be viewable in the volatility plot.
References
forum xref

My guess is that some vega lookup is using the Javascript dot accessor (data.peanut), when it should be using the square bracket lookup (data['peanut']). The problem here is when you have a period in the key: data.peanut.butter ({data: { peanut: { butter: 'yummy!' } } }) vs data['peanut.butter'] ({data: { 'peanut.butter': 'peanut butter jelly time!' } }).

Hey there @nbokulich and @thermokarst,
After reviewing this issue with @ebolyen and @Oddant1, it seems that a newer version of Vega has solved the issue. The Vega version used by the user was version 3.3.1. The current version in the QIIME 2 source code is version 4.4.0. I replicated this issue (as closely as I could) with current files from the tutorial, and Q2 View has no issue displaying the plot for group column names containing a..
TLDR: I think we can close this issue.
Thanks.

thanks for investigating @David-Rod !
Maybe we should a) pin a minimum vega version (if it is not already) and b) maybe also toss in a quick unit test? (though if we are positive this is related to the vega version then no need for a test). Interested to hear @thermokarst 's thoughts.


remove external service dependency when running async tests

Previously we depend on waithook.com to verify the callback when calling
sendTransactionAsync API. However the service is not stable and
currently down causing tests failing.
This is to use the same codebase used in waithook.com and run in a
container to provide same mechanism used in the test

Does this mean that running with an existing quorum-examples network won't work (as the tests will try to connect to an inexistent docker container/service)?
Maybe you can add the docker commands required to start/stop a local waithook container to the docs.


move getChart  to public

chart  "bitnami/contour"  have some CRDs under the  directory "resources",  not "crds", so download the chart first  and  manage the CRDs by manual is likely the prefer choice,  move the "getChart" to public, make my life easy, perhaps someone else like the idea too.
Signed-off-by: Gang Liu<EMAIL_ADDRESS>

I need this as well, need to preproces some metadata. Will take a look at this PR as well. I have also a PR that does something similar. Will abandon my in favour of this.

done

Hey @zvonkok @izturn @elenz97 !
Have a requirement for modifying the chart values, but I can't seem to figure out how an OCI chart can be pulled through this function. I constantly get an error missing registry client, even though the GetChart function is called through the same initialized client that can install OCI charts.
Do you guys have any examples of calling this function so that it works for OCI charts?
Thanks!


hljs.COMMENT gives the word 'a' the relevance 1

Use of the recommended function hljs.COMMENT leads to every occurrence of the word a in any comment to be given the relevance 1. I don't see how this makes any sense.
How to reproduce:
Create a new language definition file, say src/languages/test.js, with the following content:
export default function(hljs) {
  return {
    name: 'testlanguage',
    contains: [
      hljs.COMMENT(
        '#',
        '$',
        {
          relevance: 0
        }
      )
    ]
  };
}

Next, build using node tools/build.js -n test and open tools/developer.js.
In the Code field, write
# a

A relevance of 1 is displayed.
If you add more as (separated by spaces), the relevance increases accordingly.
Other words do not have this effect (as far as I tried).

After a review of the code, this is clearly by design and supposed to improve language recognition.

Correct.


docs: Explain how to customize Code Dx props

:tada: This PR is included in version 1.29.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Scaling the audit table

The audit table can get quite large depending on how extensively it is used within a database.  Can you suggest any optimized ways to partition this table, i.e., what partition key to use, etc.  Or perhaps have multiple partition tables per group of tables, let's say within a particular schema?

Can you suggest any optimized ways to partition this table

I haven't had experience with optimizing query performance at that scale yet, but ts seems like the easy choice for partitioning. Im not clear how we'd allow user configurability, or provide useful defaults, for partitioning with the table as part of an extension but its a good question
I'll leave this open in case others have ideas

thanks

I agree this needs partitioning to become relevant for large projects. A good practice would be to partition by table and sub-partition by year & month.
Partition creation can be automated:
Partition by Table

Create audit.record_version partitions for existing tables upon extension install
Add a trigger for creating audit.record_version partitions on 'CREATE TABLE'

Sub-partition by Year & Month

Add a function that adds a partition to each existing audit.record_version.table_oid for the current month
Chech if pg_cron is installed, only proceed if it's available
Run the function upon extension install
Schedule the function to run using pg_cron at the first day of each month (0 0 1 * *)

Partition Naming Proposal
audit.record_version.table_oid.year.month
Example: audit.record_version.16328.2023.08
P.S. The sql code is very straight-forward but I haven't dealt with pg extensions before. I could really use a contributing guide.

this project is a convenient approach for maintain an audit history for low-mid volume tables for consumption within SQL. Past that use-case, we recommend logging based solutions that are more configurable like https://www.pgaudit.org

I'm going to close this issue as out-of-scope but you're welcome to continue the discussion to suggest partitioning strategies for e.g. forks or local use!

True, pgaudit is the better choice but I wanted something more practical for Actions Auditing for my app, not a full fledged db auditing, and this extension was a big step towards that.
Anyway, for whomever is concerned I have added partitioning in my fork. This is a breaking change as you cannot add partitioning to an existing table, thus audit.record_version needs to be created from scratch. I guess you could rename the existing table and copy over the data.
The naming convention for the partitions is: audit.record_version_SCHEMA_TABLE_YEAR_MONTH e.g. audit.record_version_public_account_2023_8 but since partition pruning is enabled by default you only need to query audit.record_version.
P.S. This isn't battle tested yet; I just forked it last night. Let me know if you need an update in a few months.


Figure out if if there's a way for metered and dipstick to work with metrics-core.

Both @fralalonde and @magnet have expressed interest in making their libraries (dipstick and metered, respectively) compatible with metrics/metrics-core.
Just gonna dump some thoughts here:
Ingestion
We need to come up with an ingestion trait, similar to Recorder.  Until we have a trait, we can't reasonably expect these libraries, or others, to interoperate because then they'd just be tying themselves to metrics instead of metrics-core.
I think for dipstick this could be as easy as a new type of output scope that wraps a TheoreticalMetricIngestTrait, and I guess the inverse is also true: it could have an input scope that implements the trait.
For metered, I haven't looked closely enough at the code but given that it generates registry impls via procedural macros, I think it's possible that it could have the auto-generated registries implement the trait and use it?  Not sure if it generates a field per specific metric or stores them generically: our plain interface only really works well if you can feed it a string for the metric name and the value, rather than treating it like a field that you map directly to a callsite.
Exporting
Exporting only requires supporting SnapshotProvider and/or AsyncSnapshotProvider.
Both libraries could have their own snapshot type -- just holds all the metric names -> values, really -- and implement the trait.  They'd also need to be able to provide a snapshot provider, like Controller, to actually take the snapshot.  That type would have to be thread-safe.

In metered, it should indeed be possible to implement the trait on generated registries. There is one generated registry for each annotated impl block, which contains sub-registries for each measured method, which contains one metric per field (for each metrics applied to the said method). This allows having no complex look-up logic associated to finding a metric, lets us keep the precise type for each metric (compared to erasing them or having enums in a generic storage), which in turns makes it easy to implement whatever traits we like.

I think that's maybe what makes it hard, though? The ingest interface is basically something like update_counter(name: &str, value: u64) so there's a little bit of erasure that you can't entirely avoid.

Tobz, you are right that makes it hard to implement using a generic wrapper, but I was thinking to implement the trait on generated registries (the trait implementation would be generated as well).
You would have something like registry.record(&mut recorder) that would call, for each metric c, recorder.update_<metric_type>("<generated_metric_name>", c). We can also track whether metrics are counters, gauges or histograms.
Metered's API normally doesn't distinguish those, but nothing prevents us from implementing a trait returning a metric type.
These extra implementations could be behind feature gates.
As for generated registries, iirc there's an example in the documentation, otherwise cargo expand should help.

Similar to #16, this issue is sufficiently old that I think it would need a complete re-examination to understand where both crates could work together, and for the sake of clearing the backlog, I'm going to close this.


Underscore-prefixed fields aren't supported

According to the VSCode extension, prefixing a field with an underscore causes an error.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks so much for this PR @nelsonpecora! Since this behaviour is considered a bug that is already tracked in this GitHub issue, I just updated the docs with a note about this bug as well. I'm therefore closing this PR without merging. Thanks a lot for your help though and please don't hesitate to open further PRs or issues in the future :)
(Quick side-note though: You need to sign our CLA, otherwise I won't be able to merge your PRs).


msgpack errors when using iter() with intervals between each batch call

Good Day!
I've encountered this peculiar issue when trying to save up memory by processing the items in chunks. Here's a strip down version of the code for reproduction of the issue:
import pandas as pd

from scrapinghub import ScrapinghubClient

def read_job_items_by_chunk(jobkey, chunk=10000):
    """In order to prevent OOM issues, the jobs' data must be read in
    chunks.

    This will return a generator of pandas DataFrames.
    """

    client = ScrapinghubClient("APIKEY123")

    item_generator = client.get_job(jobkey).items.iter()

    while item_generator:
        yield pd.DataFrame(
            [next(item_generator) for _ in range(chunk)]
        )

for df_chunk in read_job_items_by_chunk('123/123/123'):
    # having a small chunk-size like 10000 won't have any problems

for df_chunk in read_job_items_by_chunk('123/123/123', chunk=25000):
    # having a bug chunk-size like 25000 will throw out errors like the one below

Here's the common error it throws:
<omitted stack trace above>

    [next(item_generator) for _ in range(chunk)]
  File "/usr/local/lib/python2.7/site-packages/scrapinghub/client/proxy.py", line 115, in iter
    _path, requests_params, **apiparams
  File "/usr/local/lib/python2.7/site-packages/scrapinghub/hubstorage/serialization.py", line 33, in mpdecode
    for obj in unpacker:
  File "msgpack/_unpacker.pyx", line 459, in msgpack._unpacker.Unpacker.__next__ (msgpack/_unpacker.cpp:459)
  File "msgpack/_unpacker.pyx", line 390, in msgpack._unpacker.Unpacker._unpack (msgpack/_unpacker.cpp:390)
  File "/usr/local/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 67: invalid start byte

Moreover, it throws out a different error when using a much bigger chunk-size, like 50000:
<omitted stack trace above>

    [next(item_generator) for _ in range(chunk)]
  File "/usr/local/lib/python2.7/site-packages/scrapinghub/client/proxy.py", line 115, in iter
    _path, requests_params, **apiparams
  File "/usr/local/lib/python2.7/site-packages/scrapinghub/hubstorage/serialization.py", line 33, in mpdecode
    for obj in unpacker:
  File "msgpack/_unpacker.pyx", line 459, in msgpack._unpacker.Unpacker.__next__ (msgpack/_unpacker.cpp:459)
  File "msgpack/_unpacker.pyx", line 390, in msgpack._unpacker.Unpacker._unpack (msgpack/_unpacker.cpp:390)
TypeError: unhashable type: 'dict'

I find that the workaround/solution for this would be to have a lower value for chunk. So far, 1000 works great.
This uses scrapy:1.5 stack in Scrapy Cloud.
I'm guessing this might have something to do with the long waiting time that happens when processing the pandas DataFrame chunk, and when the next batch of items are being iterated, the server might have deallocated the pointer to it or something.
May I ask if there might be a solution for this? since a much bigger chunk size will help with the speed of our jobs.
I've marked it as bug for now as this is quite an unexpected/undocumented behavior.
Cheers!

Hey Kevin, thanks for the report! Short question, have you used latest 2.1.1 version? There was a hidden bug up to the version which could lead to the wrong iteration behavior.

If yes, there's something else you could try. In the current implementation you try to iterate through all items at once, and assuming the amount of items is huge and processing data takes time - it's possible to reach some timeout. But as you know the desired chunk size beforehand and it's large enough, you could send a request per chunk using pagination parameters and handle the data with pandas as long as you need, something like this:
In [34]: def read_job_items_by_chunk(job, chunk=10000):
    ...:     processed = 0
    ...:     while True:
    ...:         next_key = job.key + '/' + str(processed)
    ...:         items = [item for item in job.items.iter(count=chunk, start=next_key, meta='_key')]
    ...:         yield items
    ...:         processed += len(items)
    ...:         if len(items) < chunk:
    ...:             break

Thanks @vshlapakov! The project was using 2.0.3. I'll try to use 2.1.1 to confirm if it indeed fixes up the wrong iteration behavior. I should have some results to verify in some weeks.
Otherwise, I'll try to use the the pagination suggestion you've introduced. Cheers!

Hi @vshlapakov, reporting in that version 2.1.1 still has this problem in it. I'll now proceed in trying out the pagination behavior you've suggested :)

Gotcha, thanks for the update! Let me know when you test the approach :+1:

Hi @vshlapakov, I've made a PR in #133 from your suggestion. I think having this convenient method would be really helpful in cases where we're processing a large number of items.
@manycoding, I see that this might also be of use to arche from your issue in https://github.com/scrapinghub/arche/issues/140.
Thanks!

Would it not be nicer to have it as a default (at some point) behavior in normal .iter()? I imagine people will start using it for no other reason then to avoid connection issues and everyone ends up with a useless nesting level in their code. Backward compatible alternative:
def iter(..., buffer: Optional[int] = None, in_chunks: bool = False):
    if not buffer:
        ... # proceed as usual
        return

    for chunk in self._list_iter(...):
        if in_chunks:
            # for those actually needing chunks
            yield chunk
        else:
            yield from chunk

This also addresses #135 as it would implicitly enable pagination.

I do get where you're coming from @hermit-crab, though I think iter() and iter_by_chunks() will have a different structure of items being returned. iter() will be a generator of items in this case whilst iter_by_chunks() is a generator of list of items.

Thank you for the reply @BurnzZ. Yes, I understand they will generate different structures but to elaborate more on what I mean is I believe this solution creates a situation where you have 2 distinct methods which do roughly the same thing (retrieving a resource of a job) with one of them being clearly preferable over the other despite slightly different and less commonly needed output format of the same data. At that point why would you ever use iter()? If the docs will mention on iter() that in case your data is large (TODO define large), to avoid memory and connection issues see iter_by_chunks(), then the most reasonable thing a user would do is to just use that method right off the bat to avoid dealing with issues later when their data may grow. Even if that means extra nesting level that they won't need.
For instance on the issue mentioned above https://github.com/scrapinghub/shub-workflow/issues/5, the solution will end up being something like in here:
    def _process_job_items(self, scrapername, spider_job):
        first_keyprefix = None
        items_gen = (item for chunk in spider_job.items.iter_in_chunks() for item in chunk)
        # or any other variation of flattening a list
        for item in items_gen:

While I think it would be nicer to just provide a flag to spider_job.items.iter() to enable a chunk read or for it to be enabled implicitly via a flag to the sh client on creation.
That would be similar style to how pandas do it to allow for io/memory efficient reads with chunksize parameter on their read_*() methods. For pandas tho they don't have a choice but to start generating chunks since DataFrame/Series are more or less immutable objects that you cannot easily stream. In case if iter() we can abstract the buffer away and provide a flag if it's really needed.

@hermit-crab That makes sense to me, the methods are very close to each other, the only major difference is the output format, while the iter_by_chunks "pagination" logic would be preferable in a default case in the long term. At the same time, I would try avoiding the situation when the public iter method returns a different result type depending on input, but it should be solvable via a common internal helper serving both iter and list_iter.
I'm going to close this issue as the original problem is solved, but I'm looking forward to improve it if possible, when we agree on the implementation.

Hi @BurnzZ, @hermit-crab, @vshlapakov, @Gallaecio,
I have observed a situation where using iter() with even a low value of count=1000 is causing msgpack errors in this job: https://app.zyte.com/p/435191/873/1150/log?line=35838.
You can see in the urllib3 debug log that it's making the following API call in the backend: [stderr] 2023-08-13 13:18:29 [urllib3.connectionpool] DEBUG: [https://storage.scrapinghub.com:443](https://storage.scrapinghub.com/) "GET /items/435191/897/44?start=435191%2F897%2F44%2F35697000&count=1000&meta=_key HTTP/1.1" 200 None. Worth mentioning here that even though count=1000 but the start value is huge as the job is processing around 43M items in chunks of 1000.
Would converting the iterator to a list help solve this issue? Let me know if I should open up an issue for this. Thanks.


nixos/tests/gnupg: init

Description of changes
This adds a test for various GnuPG functionalities. It ill be useful since
NixOS module is expected to undergo some changes soon.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested with gnupg.tests
[x] Tested compilation of all packages that depend on this change (gnupg)
[x] Tested basic functionality of all binary files (not needed)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

cc: @SuperSandro2000

Forgot to remove a commit from the branch.

@tfc Thank you!

Tested with gnupg.tests

I don't think this is good enough. We should build a test suite for the test framework that doesn't involve running all NixOS tests, because that's just not feasible to do for every change to the test framework.


Add ACs for market network behaviour after parameter changes

In order to write QA tests to prove that the network behaviour is as expected after a network parameter change we need to add ACs to the relevant specifications for the following network parameters:

[ ] market.liquidity.targetstake.triggering.ratio
[ ] market.margin.scalingFactors
[ ] market.fee.factors.makerFee
[ ] market.fee.factors.infrastructureFee
[ ] rewards.marketCreationQuantumMultiple

market.liquidity.targetstake.triggering.ratio is copied over to a market and becomes a market parameter

market.margin.scalingFactors should be copied over to a market and become a market parameter

market.fee.factors.makerFee  should be copied over to a market and become a market parameter

market.fee.factors.infrastructureFee must be network wide

rewards.marketCreationQuantumMultiple must be network wide

### Network wide
If market.fee.factors.infrastructureFee is changed via governance then all trades that execute after the governance proposal is enacted charge the new infrastructure fee.
If rewards.marketCreationQuantumMultiple is changed via governance the change takes effect immediately and the new value is used at the end of the epoch to decide if market creators are eligible for reward.

### Updates in market change proposals
Once market.fee.factors.makerFee is changed all subsequent trades incur the new maker fee.
Once market.liquidity.targetstake.triggering.ratio is changed, next time conditions for entering auction are evaluated the new triggering ratio is applied.
Once market.margin.scalingFactors are updated, next time any margin account balance is considered the new scaling factors are applied.


Add email attachment support

Any change ?

Hi @raymand211092 are you interested on this feature?
It's been a while since I looked at this bridge. Need to bring a lot of dependencies upto date as well.


qt5: Upgrade to Qt 5.13.0

Update recipes for Qt 5.13.0, add new recipes for Qt modules
qtcoap and qtlottie, and remove qtcanvas3d.
Signed-off-by: Samuli Piippo<EMAIL_ADDRESS>

I've added this to master-next, but seems to break:
maliit-framework-qt5 http://errors.yoctoproject.org/Errors/Build/83901/
maliit-plugins-qt5 http://errors.yoctoproject.org/Errors/Build/83902/
qt5-creator http://errors.yoctoproject.org/Errors/Build/83904/
qt5-opengles2-test http://errors.yoctoproject.org/Errors/Build/83900/
Please check them.

Also some of the musl qtwebengine patches don't apply in this version, I've updated them here:
https://github.com/meta-qt5/meta-qt5/commit/c8b3085aa4b0030d6ef07b4703763692c2c58ec0
@kraj can you please check master-next with your musl builds? Thanks

Thanks for letting me know I will rebase my mut Branch and see how it pans out

@shr-project we need this patch https://github.com/meta-qt5/meta-qt5/pull/219 on top to get qtwenengine working on aarch64, I built it for aarch64/musl/clang and all looks ok

fix for creator build: https://github.com/meta-qt5/meta-qt5/pull/221
qt5-opengles2-test fails since gles2 is not enabled, but this is already excluded from world build

https://github.com/meta-qt5/meta-qt5/pull/220 is needed to fix qttools regression with clang

In master now, thanks.
6463beab..53500ade  master -> master


Fixed movie count from 13000 to 1300

Type of PR

[x] Documentation changes
[ ] Code changes
[ ] Test changes
[ ] CI-CD changes
[ ] GitHub Template changes

Purpose of PR
Noticed an error. Not sure yet if we accept adhoc changes like this not attached to an issue.
Review notes
Issues Closed or Referenced
None

No problem when it's a small change -- especially w/ docs.  Thanks for fixing!


Issue when trying to refresh alarm status

Hi,
When I try to refresh the alarm status I get this error. What's wrong ?

Oh, this feature still in progress. Will work on that. Sorry.

@guerrerotook I add the parameter of my installation_id on the call (I get it from securitas App), it will work?


Running in Python 3 attempts to use raw_input

Hi,
I ran into this issue while trying out mbed-cli 1.9.1 on python3.5. Mbed-OS version 5.11.5.
Script fails in the following way:
> mbed publish
[mbed] WARNING: If you're using Python 3 with Mbed OS 5.8 and earlier versions, Python errors will occur when compiling, testing and exporting
---
[mbed] Working path "/home/aramanen/work/mbedtest" (program)
[mbed] Checking for local modifications...
[mbed] Uncommitted changes in program "mbedtest" in "/home/aramanen/work/mbedtest"
[mbed] ERROR: Unknown Error: name 'raw_input' is not defined
---
[mbed] WARNING: Using Python 3 with Mbed OS 5.8 and earlier can cause errors with compiling, testing and exporting
---

The error is on this this line, and fails since raw_input() is input() in py3.
    if repo.dirty():
        action("Uncommitted changes in %s \"%s\" in \"%s\"" % (repo.pathtype(repo.path), repo.name, repo.path))
        if msg:
            repo.commit(msg)
        else:
            if sys.version_info[0] == 3:
                input('Press enter to commit and publish: ')
            else:
                raw_input('Press enter to commit and publish: ')
            repo.commit()

Seems to make it work fine.

Steps to reproduce:

Install module via pipenv
clone blinky
Modify two files in the repository
Commit one of them via git, leave the other as unstaged
run mbed publish

Should mbed-cli be able to handle this situation?
Sorry about the horrible shell formatting.

 ~/tmp  mkdir mcl2
 ~/tmp  cd mcl2
 ~/t/mcl2  pipenv --three
Creating a virtualenv for this project…
Pipfile: /home/aramanen/tmp/mcl2/Pipfile
Using /usr/bin/python3.5m (3.5.2) to create virtualenv…
⠋ Creating virtual environment...Using base prefix '/usr'
New python executable in /home/aramanen/.local/share/virtualenvs/mcl2-BkELMTBZ/bin/python3.5m
Also creating executable in /home/aramanen/.local/share/virtualenvs/mcl2-BkELMTBZ/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python3.5m

✔ Successfully created virtual environment!
Virtualenv location: /home/aramanen/.local/share/virtualenvs/mcl2-BkELMTBZ
Creating a Pipfile for this project…
 ~/t/mcl2  pipenv install mbed-cli==1.9.2
Installing mbed-cli==1.9.2…
Adding mbed-cli to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (1ca9e4)!
Installing dependencies from Pipfile.lock (1ca9e4)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
 ~/t/mcl2  pipenv shell
Launching subshell in virtual environment…
 ~/t/mcl2   source /home/aramanen/.local/share/virtualenvs/mcl2-BkELMTBZ/bin/activate.fish
(mcl2)  mcl2-BkELMTBZ  ~/t/mcl2  mbed-cli --version
1.9.2
(mcl2)  mcl2-BkELMTBZ  ~/t/mcl2  mbed import https://github.com/arskanov/mbed-os-example-blinky.git
[mbed] WARNING: If you're using Python 3 with Mbed OS 5.8 and earlier versions, Python errors will occur when compiling, testing and exporting
---
[mbed] Working path "/home/aramanen/tmp/mcl2" (directory)
[mbed] Importing program "mbed-os-example-blinky" from<EMAIL_ADDRESS>at latest revision in the current branch
[mbed] Adding library "mbed-os" from<EMAIL_ADDRESS>at rev #51d55508e840
[mbed] Auto-installing missing Python modules (colorama, urllib3, prettytable, junit_xml, pyyaml, jsonschema, future, six, mbed_cloud_sdk, requests, idna, pyserial, jinja2, intelhex, intervaltree, mbed_ls, mbed_host_tests, mbed_greentea, beautifulsoup4, fuzzywuzzy, pyelftools, manifest_tool, icetea)...
(mcl2)  mcl2-BkELMTBZ  ~/t/mcl2 
(mcl2)  mcl2-BkELMTBZ  ~/t/mcl2  cd mbed-os-example-blinky/
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master   echo "#BIG" >> README.md
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  echo "// Comment" >> main.cpp
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  git add main.cpp
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  git commit -m "Add comment"
[master e819067] Add comment
 1 file changed, 1 insertion(+)
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  gs
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")
(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  mbed status
[mbed] WARNING: If you're using Python 3 with Mbed OS 5.8 and earlier versions, Python errors will occur when compiling, testing and exporting
---
[mbed] Working path "/home/aramanen/tmp/mcl2/mbed-os-example-blinky" (program)
[mbed] Status for "mbed-os-example-blinky":
 M README.md

(mcl2)  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ±  mbed publish
[mbed] WARNING: If you're using Python 3 with Mbed OS 5.8 and earlier versions, Python errors will occur when compiling, testing and exporting
---
[mbed] Working path "/home/aramanen/tmp/mcl2/mbed-os-example-blinky" (program)
[mbed] Checking for local modifications...
[mbed] Uncommitted changes in program "mbed-os-example-blinky" in "/home/aramanen/tmp/mcl2/mbed-os-example-blinky"
[mbed] ERROR: Unknown Error: name 'raw_input' is not defined
---
[mbed] WARNING: Using Python 3 with Mbed OS 5.8 and earlier can cause errors with compiling, testing and exporting
---
(mcl2)  ✘  mcl2-BkELMTBZ  ~/t/m/mbed-os-example-blinky   master ± 

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-1006

Resolved by #870 Good job!

Thanks!


About Mapper "color_2_depth_space"

Hello, when I use mapper, the color_2_depth_space containing imshow will not have a window display.
Can you help me? Thank you

Can you show me your code ? Did you pass the parameter show=True to the function ?


Module make observer

Giving continuity to PR#915.
Please give me feedback on what needs to be changed and I'll change it as soon as possible. I think this is a must have on this project.

Hey, Is there anything more I can do? All checks have passed and there are no conflicts.

Hi.
Same comments as the other PR apply here too, I see those were not addressed.


Adding tunisia to coutnry list

Adding tunisia to country list hopping that github consider us as a faithful users for github ❤

Why:
Closes ISSUE

What's being changed (if available, include any code snippets, screenshots, or gifs):

Check off the following:

[ ] I have reviewed my changes in staging (look for the "Automatically generated comment" and click the links in the "Preview" column to view your latest changes).
[ ] For content changes, I have completed the self-review checklist.

@bassem97 Thanks for opening an issue! We ❤ all of our users and appreciate you reaching out from Tunisia. 💖
The article linked in the issue is specifically for our SMS support in our two-factor authentication process. We’re only able to offer SMS authentication in countries where our SMS providers can guarantee a high deliverability rate. I’m sorry to say their service is not reliable enough in Tunisia at this point in time, so we’re unable to offer SMS authentication as an option for users in Tunisia.
While we won’t be able to offer SMS as an option for you, you might consider enabling two-factor authentication using a TOTP application. If you’re interested, we have a guide that will walk you through setting this up:
Configuring two factor authentication using a totp mobile app
With 2FA enabled, be sure to set up some of these additional fallback recovery methods as well:

Download your recovery codes. This is by far the best way to make sure you don’t get locked out of your account.
Add a security key. You’ll still have access to your account if you have a FIDO U2F security key added to your account.
See "Configuring two factor authentication recovery methods"

I apologize for the inconvenience, and rest assured, we’re actively working to improve the situation. 👍

@bassem97 Thanks for opening an issue! We ❤ all of our users and appreciate you reaching out from Tunisia. 💖
The article linked in the issue is specifically for our SMS support in our two-factor authentication process. We’re only able to offer SMS authentication in countries where our SMS providers can guarantee a high deliverability rate. I’m sorry to say their service is not reliable enough in Tunisia at this point in time, so we’re unable to offer SMS authentication as an option for users in Tunisia.
While we won’t be able to offer SMS as an option for you, you might consider enabling two-factor authentication using a TOTP application. If you’re interested, we have a guide that will walk you through setting this up:
Configuring two factor authentication using a totp mobile app
With 2FA enabled, be sure to set up some of these additional fallback recovery methods as well:

Download your recovery codes. This is by far the best way to make sure you don’t get locked out of your account.
Add a security key. You’ll still have access to your account if you have a FIDO U2F security key added to your account.
See "Configuring two factor authentication recovery methods"

I apologize for the inconvenience, and rest assured, we’re actively working to improve the situation. 👍

Thanks for you answer, i really appreciate it , but why other websites can use SMS authentication in tunisia while github is struggling ?


epoll_wait implementation can silently produce wrong results

This is to track the concerns I raised in https://github.com/rust-lang/miri/pull/2764#discussion_r1105055481: currently it is possible to run Rust code in Miri that calls epoll_wait and the call returns, but its behavior is just wrong. That should never happen. The call should either halt execution (e.g. via throw_unsup) or return with correct behavior. It's okay to only support a tiny fragment of what epoll_wait can do, but in that case it is IMO mandatory to fail loudly when the code being run falls outside that fragment.
This looks like it might also affect the write implementation for Event.
@DebugSteven can you look into adding the necessary checks in epoll_wait and Event::write?

Yeah I can do that. I've been working on checks in Event::write already. I'll put a PR with that progress in a few hours & work on adding checks for epoll_wait today.

https://github.com/rust-lang/miri/pull/2838 fixes the immediate issue here, I think.
Event::write still seems wrong (it uses from_be_bytes instead of the current target endianess), but I don't know if there is any way that programs could observe this.

Hi,
I don't know if this belongs here for sure but when I try to cargo +nightly miri run my project I face this backtrace:
Preparing a sysroot for Miri (target: x86_64-unknown-linux-gnu)... done
WARNING: Ignoring `RUSTC_WRAPPER` environment variable, Miri does not support wrapping.
    Finished dev [unoptimized + debuginfo] target(s) in 0.16s
     Running `/home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo-miri runner target/miri/x86_64-unknown-linux-gnu/debug/mqtt-proxy`
error: unsupported operation: returning ready events from epoll_wait is not yet implemented
   --> /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.10/src/sys/unix/selector/epoll.rs:97:9
    |
97  | /         syscall!(epoll_wait(
98  | |             self.ep,
99  | |             events.as_mut_ptr(),
100 | |             events.capacity() as i32,
101 | |             timeout,
102 | |         ))
    | |__________^ returning ready events from epoll_wait is not yet implemented
    |
    = help: this is likely not a bug in the program; it indicates that the program performed an operation that the interpreter does not support
    = note: BACKTRACE:
    = note: inside `mio::sys::unix::selector::epoll::Selector::select` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.10/src/sys/unix/mod.rs:7:28: 7:49
    = note: inside `mio::poll::Poll::poll` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.10/src/poll.rs:426:9: 426:61
    = note: inside `tokio::runtime::io::driver::Driver::turn` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/io/driver.rs:149:15: 149:47
    = note: inside `tokio::runtime::io::driver::Driver::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/io/driver.rs:122:9: 122:32
    = note: inside `tokio::runtime::signal::Driver::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/signal/mod.rs:92:9: 92:29
    = note: inside `tokio::runtime::driver::IoStack::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/driver.rs:173:40: 173:54
    = note: inside `tokio::runtime::time::Driver::park_internal` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/time/mod.rs:207:21: 207:46
    = note: inside `tokio::runtime::time::Driver::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/time/mod.rs:148:9: 148:41
    = note: inside `tokio::runtime::driver::TimeDriver::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/driver.rs:329:55: 329:74
    = note: inside `tokio::runtime::driver::Driver::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/driver.rs:69:9: 69:32
    = note: inside `tokio::runtime::scheduler::multi_thread::park::Inner::park_driver` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/park.rs:184:9: 184:28
    = note: inside `tokio::runtime::scheduler::multi_thread::park::Inner::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/park.rs:117:13: 117:50
    = note: inside `tokio::runtime::scheduler::multi_thread::park::Parker::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/park.rs:67:9: 67:32
    = note: inside `tokio::runtime::scheduler::multi_thread::worker::Context::park_timeout` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:723:13: 723:50
    = note: inside `tokio::runtime::scheduler::multi_thread::worker::Context::park` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:693:24: 693:53
    = note: inside `tokio::runtime::scheduler::multi_thread::worker::Context::run` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:544:21: 544:36
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:491:21: 491:33
    = note: inside `tokio::runtime::context::scoped::Scoped::<tokio::runtime::scheduler::Context>::set::<{closure@tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure#0}}, ()>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/scoped.rs:40:9: 40:12
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context.rs:176:26: 176:47
    = note: inside `std::thread::LocalKey::<tokio::runtime::context::Context>::try_with::<{closure@tokio::runtime::context::set_scheduler<(), {closure@tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure#0}}>::{closure#0}}, ()>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:270:16: 270:31
    = note: inside `std::thread::LocalKey::<tokio::runtime::context::Context>::with::<{closure@tokio::runtime::context::set_scheduler<(), {closure@tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure#0}}>::{closure#0}}, ()>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:246:9: 246:25
    = note: inside `tokio::runtime::context::set_scheduler::<(), {closure@tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure#0}}>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context.rs:176:9: 176:48
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:486:9: 497:11
    = note: inside `tokio::runtime::context::runtime::enter_runtime::<{closure@tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}}, ()>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16: 65:38
    = note: inside `tokio::runtime::scheduler::multi_thread::worker::run` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:478:5: 498:7
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/worker.rs:447:45: 447:56
    = note: inside `<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}> as std::future::Future>::poll` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/blocking/task.rs:42:21: 42:27
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/core.rs:328:17: 328:37
    = note: inside `tokio::loom::std::unsafe_cell::UnsafeCell::<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>>>::with_mut::<std::task::Poll<()>, {closure@tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll::{closure#0}}>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/loom/std/unsafe_cell.rs:16:9: 16:24
    = note: inside `tokio::runtime::task::core::Core::<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/core.rs:317:13: 329:15
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/harness.rs:485:19: 485:38
    = note: inside `<std::panic::AssertUnwindSafe<{closure@tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::{closure#0}}> as std::ops::FnOnce<()>>::call_once` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9: 272:19
    = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<{closure@tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::{closure#0}}>, std::task::Poll<()>>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552:40: 552:43
    = note: inside `std::panicking::r#try::<std::task::Poll<()>, std::panic::AssertUnwindSafe<{closure@tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::{closure#0}}>>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:516:19: 516:81
    = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<{closure@tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::{closure#0}}>, std::task::Poll<()>>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:142:14: 142:33
    = note: inside `tokio::runtime::task::harness::poll_future::<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/harness.rs:473:18: 488:8
    = note: inside `tokio::runtime::task::harness::Harness::<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/harness.rs:208:27: 208:55
    = note: inside `tokio::runtime::task::harness::Harness::<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/harness.rs:153:15: 153:32
    = note: inside `tokio::runtime::task::raw::poll::<tokio::runtime::blocking::task::BlockingTask<{closure@tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{closure#0}}>, tokio::runtime::blocking::schedule::BlockingSchedule>` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/raw.rs:271:5: 271:19
    = note: inside `tokio::runtime::task::raw::RawTask::poll` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/raw.rs:201:18: 201:41
    = note: inside `tokio::runtime::task::UnownedTask::<tokio::runtime::blocking::schedule::BlockingSchedule>::run` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/task/mod.rs:453:9: 453:19
    = note: inside `tokio::runtime::blocking::pool::Task::run` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/blocking/pool.rs:159:9: 159:24
    = note: inside `tokio::runtime::blocking::pool::Inner::run` at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/blocking/pool.rs:513:17: 513:27
    = note: inside closure at /home/andrey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/blocking/pool.rs:471:13: 471:54
    = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<{closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:155:18: 155:21
    = note: inside closure at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:529:17: 529:78
    = note: inside `<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>::{closure#1}::{closure#0}}> as std::ops::FnOnce<()>>::call_once` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9: 272:19
    = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>::{closure#1}::{closure#0}}>, ()>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552:40: 552:43
    = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>::{closure#1}::{closure#0}}>>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:516:19: 516:81
    = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>::{closure#1}::{closure#0}}>, ()>` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:142:14: 142:33
    = note: inside closure at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:528:30: 530:16
    = note: inside `<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tokio::runtime::blocking::pool::Spawner::spawn_thread::{closure#0}}, ()>::{closure#1}} as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5: 250:71
    = note: inside `<std::boxed::Box<dyn std::ops::FnOnce()> as std::ops::FnOnce<()>>::call_once` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9: 2015:52
    = note: inside `<std::boxed::Box<std::boxed::Box<dyn std::ops::FnOnce()>> as std::ops::FnOnce<()>>::call_once` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9: 2015:52
    = note: inside `std::sys::unix::thread::Thread::new::thread_start` at /home/andrey/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:108:17: 108:64
    = note: this error originates in the macro `syscall` (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to 1 previous error

That's just https://github.com/rust-lang/miri/issues/602 -- we don't currently support epoll in Miri.


Port pyright optimization that skips loop flow nodes that dont contain relevant references

Fixes #51525
This required reworking how we compare references so we could equate them within a set - that change, in and of itself, may actually also be perf positive (since we now get to cache the walk over the reference's AST), but both changes together is somewhere between neutral and 4% gains in local testing, depending on the suite. I'll need to see if this bears out on the much slower CI box, though, since 4% locally is < 0.1s, which is well in the realm of "OS scheduler variance" for some of these tests on my box. Do note, unlike pyright, which in the linked issue's change, calculates the references used within a loop during binding, we calculate it late on-demand and cache it in the checker. This is basically required for us, since we equate references using the identity of the resolved symbol of the root node of the reference (which... may be a bit circular with expression checking!). This means we don't get to collect the references for "free" while doing the binder AST walk; but given how often the set of references is referred to by control flow, it should probably still be an improvement in all but pathological cases (eg, thousands of references within a loop, none outside of it).

@typescript-bot perf test this

This is basically required for us, since we equate references using the identity of the resolved symbol of the root node of the reference

Can you give an example of where this comes up? Is it for a computed reference like this?
const propName = "a";
obj[propName];

More like to support block scoped variable shadowing -
let x: 1 | 2 = 1;
x; // narrowed to `1`
{
  let x = 2
  x; // only the x within this block - `number`
}
x; // still `1`

which is probably the core reason it uses symbol identity for the roots and not lookup names, and then gets further complicated when the x is export (since that export can have other things merged into it by augmentations, so that identity can be changed by the symbol merging process). Presumably it's this way because it simplifies scope handling when comparing references, since you don't need to check that the references share the same scope or aren't being shadowed by a new reference.
It's mostly speculation on my part, since that's just how it's always been, at least as far as I've known. We've always compared symbols when looking at the root identifiers of references, and not the identifier text itself.

squints +5% in TFS, -1% in compiler-unions. OK, perf testing on my local machine is useless; results are nothing like the CI box. System noise is a lot worse when the tests run an order of magnitude faster. Guess I'm pushing all my iterations on this. Since there's a small perf improvement to compiler-unions - the test most likely to see improvements - this is likely a game to see if I can keep those improvements without a cost to the other suites. Then again, when you proposed this @DanielRosenwasser @jakebailey you were hoping for significantly more obvious gains? It's possible this work is only a minor improvement on top of the flow loop cache we already have in more normal code.

i was hoping for obvious gains but my memory may be so poor that maybe this only manifests in examples with lots and lots of stuff going on.
will test with my own perf machine to verify as this is very weird
@typescript-bot perf test faster

I haven't gotten any version of this to have an outsized, consistent effect locally on our test suite; I'm going to assume our existing flow loop cache largely subsumes this, making it much less impactful.


Unable to use ipad as a secondary display

I tried all the troubleshooting steps, but my windows is just mirroring the display, and 'Extend' doesn't seem to work. What should I do?

Can you confirm that the virtual display is created correctly in the system settings?
You can also try https://github.com/ClassicOldSong/Apollo/releases/tag/apollo-0.0.0-preview to see if it works for you.

You need to install Moonlight on your iPad in order to use Sunshine/Apollo.

I use Sunshine and Moonlight on the iPad.

Then you need to start streaming on your iPad using Moonlight.

Yes i did that and its only mirroring the windows screen; even when I click extend. Ive also tried the Troubleshooting method for the same problem, but no changes still.

So have you seen any virtual screen created temporarily when the stream starts?
Or can you try Apollo as I mentioned above to see if it works?

I do have Parsec VDD installed...but i dont know how to see if any virtual screen is created or not.

Could you just try Apollo and see if it works?
By the way what version of  Windows are you using?

@ClassicOldSong I have the same issue.
Apollo only mirrors the PC screen to my ipad
I am using windows 11
And when I open display settings, I only see the screen of my laptop, so there is no virtual screen displayed

If you have seen 1|2 in your Windows settings, then the only thing you need to do is make the second display Extended
If not, you may have not got the vdd driver installed correctly.
If virtual displays can create but you're not able to launch any other apps, it's probably a permission issue: https://docs.lizardbyte.dev/projects/sunshine/en/latest/troubleshooting/windows.html
And this project abandoned as described in the README, if you have issues with Apollo please open an issue there instead of here.


Dictionary/Words/Compound/Term[@CompoundAlternate] Does Not Work

In a legacy application, we have a lot of occurrences of the term "Username" (lower case "n"), which correctly causes CA1702 errors/warnings. i.e.:
warning CA1702: Microsoft.Naming : The discrete term 'Username' in member name '[redacted]' should be expressed as a compound word, 'UserName'
Adding this:
<Term CompoundAlternate="Username">username</Term>
to the Dictionary/Words/Compound section, the expectation is that "Username" should no longer cause CA1702 warnings/errors.
However, adding the CompundAlternate only changes the message to this:
warning CA1702: Microsoft.Naming : The discrete term 'Username' in member name '[redacted]' should be expressed as a compound word, 'Username'.

Note the lower case "n" in both instances of the term "Username".
Refactoring all occurrences of "Username" is not feasible, ans suppressing literally thousands of CA violations is not desirable.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5c0c5937-e895-d2e3-0e14-655cf4e52d6a
Version Independent ID: abe9dea4-c830-c476-1e9d-fd2386d014c3
Content: How to: Customize the Code Analysis Dictionary - Visual Studio
Content Source: docs/code-quality/how-to-customize-the-code-analysis-dictionary.md
Product: visual-studio-windows
Technology: vs-ide-code-analysis
GitHub Login: @gewarren
Microsoft Alias: gewarren

@MadQ Thank you for the feedback.
This is product-related and might be better suited for the Roslyn issues queue.
@gewarren Can the issue be transferred there?

My suggestion is to add username as a Recognized word instead of as a CompoundAlternate.
Note that this rule CA1702 is deprecated for FxCop analyzers.

@gewarren https://github.com/gewarren I actually did try adding it as
a Recognized word, with the same result. I tried quite a few other things,
which are probably not relevant. Since it is a legacy application, we try
to touch it as little as possible; I just happened to be updating the
ruleset for the TFS project, and was trying to minimize the number of build
warnings. As it stands now, there are so many warnings that the important
ones are being lost in the noise. We probably cannot justify the time to
switch that one solution over to FxCop analyzer, which I am doing for all
the active ones.
Thank you all for the replies. For the most part, I just wanted to add the
issue to the documentation to possibly save others the time, effort and
frustration I went through. Due to binary CA being deprecated, I don't
really expect it to be fixed.
-- Quinten Martens
On Fri, Sep 6, 2019 at 5:51 PM Genevieve Warren<EMAIL_ADDRESS>wrote:

My suggestion is to add username as a Recognized word
https://docs.microsoft.com/en-us/visualstudio/code-quality/how-to-customize-the-code-analysis-dictionary?view=vs-2017#BKMK_DictionaryWordsRecognizedWord
instead of as a CompoundAlternate.
Note that this rule CA1702 is deprecated
https://github.com/dotnet/roslyn-analyzers/issues/711 for FxCop
analyzers.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/visualstudio-docs/issues/3875?email_source=notifications&email_token=AA6226KVTRCPYKQQRZWMRBTQILGEVA5CNFSM4IULMEJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6EEZDY#issuecomment-529026191,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA6226PLBB3HUG2RBIH5MSTQILGEVANCNFSM4IULMEJA
.

Ah okay, sorry that didn't work either.
Since this is a product issue, you'd need to log it through "Report a Problem" in Visual Studio. However, as you noted, it's not likely to be fixed at this point.
I'm closing this issue now, but it will still appear associated with the referenced topic.


Language pt-BR.

Hello, this is a good module, great.
I made a translation into pt-BR.
pt-BR.json

Ty for the contribution


Failed to start Docker - storage.journal.enabled

Hey,
I run Rocketchat including MongoDB on my Unraid server.
Since a few days I have problems to start MongoDB. Perhaps you can help me further.
I updated the two Dockers and since then MongoDB no longer starts with the following error message:
Unrecognized option: storage.journal.enabled try 'mongod --help' for more information
This error message also occurs during a new installation.
Unfortunately I can not find a solution to start the database again.
Thank you in advance.

Maybe you upgraded to a newer version where this option is no longer present? https://stackoverflow.com/a/76924946

From Mongo: https://www.mongodb.com/docs/manual/core/journaling/#:~:text=Starting in MongoDB 6.1%2C journaling,--nojournal command-line options.

Starting in MongoDB 6.1, journaling is always enabled. As a result, MongoDB removes the storage.journal.enabled option and the corresponding --journal and --nojournal command-line options.


Crawler failed, because of ValueError.

Crawling course for 105|10
Traceback (most recent call last):                                                                                |
  File "manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/root/.venv/tmp/lib/python3.5/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/root/.venv/tmp/lib/python3.5/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/root/.venv/tmp/lib/python3.5/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/root/.venv/tmp/lib/python3.5/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/root/git/test/NTHU_Course/crawler/management/commands/crawl_course.py", line 42, in handle
    crawl_course(ACIXSTORE, auth_num, cou_codes, ys)
  File "/root/git/test/NTHU_Course/crawler/crawler.py", line 111, in crawl_course
    handle_curriculum_html(response.text, cou_code)
  File "/root/git/test/NTHU_Course/crawler/crawler.py", line 95, in handle_curriculum_html
    collect_class_info(tr, cou_code_stripped)
  File "/root/git/test/NTHU_Course/crawler/crawler.py", line 71, in collect_class_info
    course_dict = course_from_tr(tr)
  File "/root/git/test/NTHU_Course/crawler/course.py", line 126, in course_from_tr
    part['size_limit'], part['fr'] = get_slfr(extract_text(tds[6]))
  File "/root/git/test/NTHU_Course/crawler/course.py", line 75, in get_slfr
    sl = int(sl)
ValueError: invalid literal for int() with base 10: '25新生保留25人'

cc @afg984

Is this still the case?

https://github.com/henryyang42/NTHU_Course/blob/upgrade_py3_django110/crawler/course.py#L72

10510CHEM345002
Traceback (most recent call last):                                                                                |
  File "/Users/henryyang/Desktop/NTHU_Course/crawler/crawler.py", line 51, in save_syllabus
    course_dict = course_from_syllabus(html)
  File "/Users/henryyang/Desktop/NTHU_Course/crawler/course.py", line 161, in course_from_syllabus
    document = lxml.html.fromstring(html)
  File "/Users/henryyang/Desktop/NC/lib/python3.5/site-packages/lxml/html/__init__.py", line 867, in fromstring
    doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
  File "/Users/henryyang/Desktop/NC/lib/python3.5/site-packages/lxml/html/__init__.py", line 752, in document_fromstring
    value = etree.fromstring(html, parser, **kw)
  File "src/lxml/lxml.etree.pyx", line 3213, in lxml.etree.fromstring (src/lxml/lxml.etree.c:77737)
  File "src/lxml/parser.pxi", line 1830, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:116674)
  File "src/lxml/parser.pxi", line 1711, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:115220)
  File "src/lxml/parser.pxi", line 1051, in lxml.etree._BaseParser._parseUnicodeDoc (src/lxml/lxml.etree.c:109345)
  File "src/lxml/parser.pxi", line 584, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:103584)
  File "src/lxml/parser.pxi", line 694, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:105238)
  File "src/lxml/parser.pxi", line 633, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:104323)
  File "<string>", line None
lxml.etree.XMLSyntaxError: line 43: b'Attribute colspan redefined'

And new error occurred...

No longer the case in #111


Enhanced version matrix

What does this do / why do we need it?
Uses https://github.com/hashicorp/go-version to have a more understandable and maintanable semver parsing.
Check lists

[ ] Test passed
[ ] Coding style (indentation, etc)

Which issue(s) does this PR fix?
closes #47

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

17 of 17 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.1%) to 60.067%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
179

Relevant Lines:
298

💛  - Coveralls


Remove Depdendabot for Docker and reduce frequency

Description

This PR

removes Dependabot from the Docker images as they are normally pinned to the semver minor version
reduces the frequency of Dependabot from weekly to monthly to control the deluge of PRs

This seems to be the consensus in the dev team.
Testing Instructions

This is hard to test locally. We can just double-check the code against the Dependabot configuration docs.
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@obulat all the Dependabot PRs already have a "🟩 priority: low" label attached.


How to build the simplest docker image that can use gpu&gui

Hi, I try to use this tutorial to build my own docker image for Ubuntu & desktop
Docker Tips - Full Ubuntu GUI in a Container Displayed on Windows (XServer)
I use this dockerfille to build docker images
https://agiledevart.github.io/Dockerfile.lubuntu
docker build -t docker-lubuntu .

but
When I run with this image
Using the commands that the deepin system can run normally, this desktop cannot be run.
sudo x11docker --desktop --network --init=systemd  --gpu --clipboard --lang=zh_CN --share /videos   x11docker/deepin
changeto
x11docker --desktop --init=systemd docker-lubuntu
It gives an error:

Failed to start dbus.service: Access denied
See system logs and 'systemctl status dbus.service' for details.
Failed to start lightdm.service: Access denied
See system logs and 'systemctl status lightdm.service' for details.

P.s.
I didn't use your deepin's dockerfile, because I have been unable to compile a usable docker image through it. When I tried to execute it with the command one by one, I couldn't understand why it did so. (Google didn't find much information)

Hello,
let's have a look at your Dockerfile:
FROM ubuntu:latest

RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y lubuntu-desktop lightdm

RUN rm /run/reboot-required*
RUN echo "/usr/sbin/lightdm" > /etc/X11/default-display-manager
RUN echo "\
[LightDM]\n\
[Seat:*]\n\
type=xremote\n\
xserver-hostname=host.docker.internal\n\
xserver-display-number=0\n\
autologin-user=root\n\
autologin-user-timeout=0\n\
autologin-session=Lubuntu\n\
" > /etc/lightdm/lightdm.conf.d/lightdm.conf

ENV DISPLAY=host.docker.internal:0.0

CMD service dbus start ; service lightdm start

x11docker runs container applications as an unprivileged user.
CMD service dbus start ; service lightdm start would need root privileges.
That explains the error messages that you got.
Most of the setup done in the Dockerfile is not needed, x11docker will do it for you.
(With --init=systemd it starts a dbus system daemon. lightdm is disabled at all, it is not needed.)
This Dockerfile should work:
FROM ubuntu:latest
RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y lubuntu-desktop
CMD startlxde

Your already build image likely will run with both of:
x11docker --desktop docker-lubuntu startlxde

and
x11docker --desktop --init=systemd docker-lubuntu startlxde

For GPU support add option --gpu. The image needs mesa-utils to provide the free MESA graphics drivers.
RUN apt-get install -y mesa-utils

If you have a proprietary NVIDIA driver, some more setup is needed.

I didn't use your deepin's dockerfile, because I have been unable to compile a usable docker image through it.

Could you show me the terminal output from docker build?

Hi, I use this command:
FROM ubuntu:latest
RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y lubuntu-desktop
CMD startlxde

When I use this dockerfile to compile,

After I start using this command
x11docker --desktop --init=systemd docker-lubuntu

/bin/sh: 1: startlxde:not found
Maybe I have to use systemd command to start this desktop?

/bin/sh: 1: startlxde:not found

I assumed that startlxde is already available in your installation.
Please try lxsession instead.
 x11docker --desktop docker-lubuntu lxsession

startlxde is an alternative and available in package openbox-lxde-session.

i used lxsession

but

when i use
FROM ubuntu:latest
RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y openbox-lxde-session

and run
x11docker --desktop docker-lubuntu startlxde

Thank you, I probably understand that docker does not need systemd/systemctl to manage software startup.
Thank you for your help, I will try to make a dockerfile of a complete Ubuntu system by referring to your dockerfile through the existing iso.

Great that it works now!

Thank you, I probably understand that docker does not need systemd/systemctl to manage software startup.

Some desktop environments need --init=systemd, for example deepin, KDE/plasma, Gnome 3 and Cinnamon.

emmm, thanks for your answer. I tried not using systemctl but executing the binary separately
I also note here
When I use apt install -y openbox-lxde-session
I can find /usr/bin/startlxde in /usr/bin/
but I use apt install -y xubuntu-desktop
only found
root@ubuntu:/# find /usr/bin/ | grep -i xubuntu
/usr/bin/xubuntu-numlockx

The number lock key for this desktop.
but i found
/usr/bin/startxfce4
and
/usr/bin/startx
Therefore I use

x11docker --desktop --init=systemd xubuntu startxfce4

to start xubuntu
it work !!
but
x11docker --desktop --init=systemd xubuntu startx

show

but why?

startx tries to run an Xorg X server on its own. This fails in containers in general. (Sort of possible, but it is a difficult and dangerous setup.)
startxfce4 is the right choice. It uses the X server provided by x11docker.

startx tries to run an Xorg X server on its own. This fails in containers in general. (Sort of possible, but it is a difficult and dangerous setup.)
Thank you for your explanation, it has benefited me a lot.
I successfully run xubuntu

In fact, it is essentially xfce4, lol.
I will continue to explore the essence of dockerfile. I feel that I have touched a little bit of skin at the moment. Regarding complex applications, especially the interaction between docker containers, I still need to continue to learn.
Thank you again.


[$250]  IOU - Infinite spinner loading on Submit button when back online after adding stop offline

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.18-0
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4836952&group_by=cases:section_id&group_order=asc&group_id=296765
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Logs: https://stackoverflow.com/c/expensify/questions/4856
Issue reported by: Applause-Internal team
Action Performed:

Start a distance request
Enter the address to the start e.g: "45n N Beach Rd, FL"
Click on add stop
Enter the address, e.g: "Beach Access Path, Sunny Isles Beach, FL"
Click Next button
Click Submit button
Go offline
Go to transaction thread page> Click on Distance> Add a stop point e.g: "Miami Beach"> Submit
Go back online
Verify the map does not shows the route
Click on Distance again

Expected Result:
Map should show the route and Submit button should response
Actual Result:
Map does not show the route when come back online, Submit button is irresponsible
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[x] iOS: mWeb Safari
[x MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/d1c90d6d-7635-4216-b34b-c451f5f8399e

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~019eb0338dff8112cc
Upwork Job ID:<PHONE_NUMBER>695149158
Last Price Increase: 2024-08-09

Issue OwnerCurrent Issue Owner: @ZhenjaHorbach

We think this issue might be related to the #collect project.

I'll see if I can reproduce on dev now and then start debugging.

Weird, I can't really reproduce on dev. Trying on staging. I think this was fixed somehow. I even tried to reproduce outside of the p2p distance beta, and I can't, so I'm going to close this out.
https://github.com/user-attachments/assets/6abd4bd8-8d0d-43a2-87bd-720dfd08d5b7


Gradle 6 deprecation warnings

When running SpotBugs under Gradle 6, I get the following warnings:
> Task :backend:spotbugsBackend
Property 'showProgress' @Input properties with primitive type 'boolean' cannot be @Optional. This behaviour has been deprecated and is scheduled to be removed in Gradle 7.0.
Type 'SpotBugsHtmlReportImpl': setter method 'setStylesheet()' should not be annotated with: @Input. This behaviour has been deprecated and is scheduled to be removed in Gradle 7.0.

Thanks for your report! #159 fixed this problem, please wait for the v2.0.1 release.


Use Travis to check that fuzz tests can build

This should proactively prevent breaking Firestore in OSS-Fuzz.

Is this still in progress or should it be closed?

I'd like to return to it at some point -- I was getting weird Travis errors which prevented me from getting much progress in this issue. Feel free to close it if you prefer -- I can reopen it at a later point if need be.

Okay, thanks for the response. Closing for now but feel free to open it back up whenever work starts again.


Update paths for ArcGIS.xcframework

Update paths for ArcGIS.xcframework as the next release of the ArcGIS SDK now builds a binary framework.

Includes automatic project updates for Xcode 12


Version bump for Carrierwave 2.0

Since carrierwave 2.0 has been released bundler is locking to carrierwave-aws 0.4.1 since it did not have an upper level version constrain applied.
It looks like this was fixed a couple weeks ago but we need a new version published to rubygems.

Thanks for the reminder! I've released v1.4.0—just shy of two years after v1.3.0.

Thanks!

There is Carrierwave 2.1.0 and constraints don't allow it again =(


Issue 375 prevent existing username registration

Proposed Changes
Fixes #375. The forms now require both the email and the username from the user
Type of Changes
What type of change this Pull Request brings to Falko?

[X] Bugfix
[ ] New Feature

Checklist

[X] This Pull Request has a significant name.
[X] The build is okay (tests, code climate).
[X] This Pull Request mentions a related issue.

Screenshots

Nice @gabrielziegler3! Thanks for contributing!


Without lidar

Hi, @brade31919 In the paper, it is mentioned that Two-Stage architecture can be used to filter radar instead of lidar, but in the program def filter_radar_points(self, input_data): are all applied lidar data to filter radar data. Can we generate preprocessed data without using lidar data?
My english is not very good, sorry.

With reading the program, I learned a lot, Thank you very much for releasing the code.

Hi @zazgf ,
The part you mentioned is actually in the data loader and that part was used to generate different sparse patterns for the experiments in Table 2 from our paper. So you can generate (1) normal radar measurements, (2) radar points filtered by the ground truth LiDAR scans, or (3) radar points filtered by the trained classifyer, etc. In the main experiment, we use (1), so there should be no concern related to "using lidar data to do the filtering".

@brade31919 Thanks for your reply.
Does this mean that I can achieve the project function without lidar?
Thanks again.

Hi @zazgf,
It depends on the sparsifier argument passed to the dataset construction (the name itself is a bit confusing). So if the sparsifier is "radar", then the inputs contain the concatenation of images and raw radar depth maps. If it's "radar_filtered", then the inputs are composed by images and filtered radar depth maps instead.
Therefore, as you can see in README.md, the sparsifier is set to "radar".

@brade31919 Get your point, I didn't realize this before.I need pay more attention on code reading. Thanks.


add Willow by Nike Inc

Project URL

https://github.com/Nike-Inc/Willow
Description

Willow is a powerful, yet lightweight logging library written in Swift.
Checklist

[x] Only one project is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Travis CI pass
[x] Supports iOS 7 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

🎉


Update to JDK 8u141-b15.

Update to JDK 8u141-b15.
Tested with a small spring-boot based application and it started up fine according to logs, plus executing java -version in the container indicates the new version, so assume everything works 😀

@stela Thank you!

Wow, that one was quick, thought there was an error refreshing the open pull requests counter! Thanks!

I looked at https://hub.docker.com/r/frolvlad/alpine-oraclejdk8/tags/ and saw the tag 8.131.11-slim was just updated a few minutes ago, mishap applying the new 8.141.15 release?

@stela Yeah, I failed there... Cancel button worked only for the 8.131.11-full image :(

@stela I have created temporary branches to rebuild 8.131.11-cleaned and 8.131.11-slim. 8.141.15 is on its way.

How about adding your "docker push" script to github as well, then contributors get to patch that one too? And, if possible somehow, trading in the git branches for a conditional Dockerfile build, using docker build –build-arg like at https://stackoverflow.com/questions/37057468/conditional-env-in-dockerfile and letting that one control how much of the JDK is removed?

Just built my app using frolvlad/alpine-oraclejdk8:8.141.15-slim and it starts up and java -version indicates the correct version. Thanks! 😄

I don't use docker push since Docker Hub pulls and builds the images automatically on their premises, and given I cannot control the build args, I don't see a point in adding this feature into the Dockerfile (I just maintain 3 branches: master (slim), cleaned, and full).
All the tags are on Docker Hub now!


git-annex faq appears to be incorrect

I'm not sure the comparison to git annex is correct. For example "Git-annex loses all metadata when shared by Git server" isn't true, as it is possible to move files around via special remotes, and store the metadata on git-only servers (datalad, which wraps git annex, shows an example of doing this exact thing).

@aragilar  thank you for noticing that!
This comparison  is very short and not exactly correct. I've tried to clarify that. Please take a look:
Git-annex is data file centric-system where DVC is an ML workflow-centric. When a project is cloned by git clone data files won't be cloned (for both DVC and Git-annex) because files content is stored in a separate data remotes . However, DVC metafiles will be cloned correctly from any Git server and ML workflow can be easily reproduced with another data files.

A slightly reworded version:
Git-annex is a datafile-centric system whereas DVC is focused on providing a workflow for machine learning and reproducible research. When a DVC or Git-annex repository is cloned via git clone, data files won't be copied to the local machine as file content is stored in separate data remotes. However, DVC metafiles (which provide the reproducible workflow) are always included in the cloned Git repository and hence can be recreated locally with minimal effort.

Something else worth mentioning in the FAQ: I got the impression (which may be wrong) that dvc isn't fundamentally bound to git, having the option of changing the repository format (when something better than git is created) would be worth mentioning (both git annex and git lfs seem to be strongly coupled to git and git's internals).

@aragilar Thank you! Would you mind creating a PR for the documentation? there is an "Edit with Github" button on top of each page. You absolutely correct about git+dvc relationship. And it's not only because we are waiting for something better, it's also about an ability to run it on top of Hg, for example. If you see a good place for this information on that page, please feel free to put it there.

Made the change, including a note that DVC is not fundamentally bound to Git.

Sorry, been really busy and so this dropped off my todo list. The changes look good!


Fix DH_check() excessive time with over sized modulus

The DH_check() function checks numerous aspects of the key or parameters
that have been supplied. Some of those checks use the supplied modulus
value even if it is excessively large.
There is already a maximum DH modulus size (10,000 bits) over which
OpenSSL will not generate or derive keys. DH_check() will however still
perform various tests for validity on such a large modulus. We introduce a
new maxium (32,768) over which DH_check() will just fail.
An application that calls DH_check() and supplies a key or parameters
obtained from an untrusted source could be vulernable to a Denial of
Service attack.
The function DH_check() is itself called by a number of other OpenSSL
functions. An application calling any of those other functions may
similarly be affected. The other functions affected by this are
DH_check_ex() and EVP_PKEY_param_check().
CVE-2023-3446 (low severity)

Typo: vulernable

Typo: vulernable

Fixed. Also rebased to resolve trivial conflicts in CHANGES.md/NEWS.md.
Please take another look.

Rebased to fix commit message typo. I assume @paulidale's approval stands.

Approval still stands - thanks

Trivial rebase to resolve CHANGES.md/NEWS.md conflicts only. I assume approval still holds.

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

Merged to master, 3.1, and 3.0 branches. Thank you.


Customer and support team feedback: Use 'Choose columns' instead of SELECT in ODATA / CI

Hi Michael,
Multiple customers have stated using OData with Select statements in URL creates corrupted table entities.
I was reviewing a customer support incident, and thought the resolution should be updated in our docs...
Recommend reviewing content with someone who understands OData better and see if the sample code needs to be updated.
Customer verbatim:

OData Select Issue
In our Odata queries we used the select function within the URL to pick which fields we want
However, a lot of our queries were going corrupt, so I had to work with support
They noticed that the queries were using the select function and actually had a different client recently have the same issue
The fix was to remove the select from the URL and instead use the Choose Columns step in the query
After doing the corrupt table was removed

Example:
let
Source = OData.Feed("https://{pii}.crm.dynamics.com/api/data/v9.1/{entityNamePii}", null, [Implementation = "2.0"]),
#"Choose columns" = Table.SelectColumns(Source, {"columns1", "columns2", "columns3"}),

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9ff70c45-490f-f6e1-651f-5b3d67f0b50b
Version Independent ID: 9ff70c45-490f-f6e1-651f-5b3d67f0b50b
Content: OData examples for the Dynamics 365 Customer Insights APIs - Dynamics 365 Customer Insights
Content Source: ci-docs/odata-examples.md
Service: customer-insights
Sub-service: audience-insights
GitHub Login: @m-hartmann
Microsoft Alias: mhart

Ack, thanks Scott! I'll follow up offline and close the issue in the public repo.