PHOENIX-6118: Multi Tenant Workloads using PHERF

@yanxinyi @ChinmaySKulkarni

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 41s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 10 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
19m 45s
master passed

+0
hbaserecompile
28m 52s
HBase recompiled.

+1 :green_heart:
compile
0m 34s
master passed

+1 :green_heart:
checkstyle
0m 35s
master passed

+1 :green_heart:
javadoc
0m 24s
master passed

+0 :ok:
spotbugs
0m 59s
phoenix-pherf in master has 42 extant spotbugs warnings.

-0 :warning:
patch
1m  8s
Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
12m  7s
the patch passed

+0
hbaserecompile
21m 42s
HBase recompiled.

+1 :green_heart:
compile
0m 35s
the patch passed

+1 :green_heart:
javac
0m 35s
the patch passed

-1 :x:
checkstyle
0m 47s
phoenix-pherf: The patch generated 756 new + 929 unchanged - 54 fixed = 1685 total (was 983)

-1 :x:
whitespace
0m  0s
The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-1 :x:
whitespace
0m  0s
The patch 1 line(s) with tabs.

+1 :green_heart:
xml
0m 10s
The patch has no ill-formed XML file.

-1 :x:
javadoc
0m 23s
phoenix-pherf generated 25 new + 32 unchanged - 0 fixed = 57 total (was 32)

-1 :x:
spotbugs
1m 11s
phoenix-pherf generated 9 new + 41 unchanged - 1 fixed = 50 total (was 42)

_ Other Tests _

+1 :green_heart:
unit
11m 17s
phoenix-pherf in the patch passed.

+1 :green_heart:
asflicense
0m 14s
The patch does not generate ASF License warnings.

69m 45s

Reason
Tests

FindBugs
module:phoenix-pherf

Found reliance on default encoding in org.apache.phoenix.pherf.util.PhoenixUtil.buildStatement(RulesApplier, Scenario, List, PreparedStatement, SimpleDateFormat):in org.apache.phoenix.pherf.util.PhoenixUtil.buildStatement(RulesApplier, Scenario, List, PreparedStatement, SimpleDateFormat): String.getBytes()  At PhoenixUtil.java:[line 557]

input must be non-null but is marked as nullable  At IdleTimeOperationSupplier.java:is marked as nullable  At IdleTimeOperationSupplier.java:[lines 52-74]

input must be non-null but is marked as nullable  At PreScenarioOperationSupplier.java:is marked as nullable  At PreScenarioOperationSupplier.java:[lines 51-80]

input must be non-null but is marked as nullable  At QueryOperationSupplier.java:is marked as nullable  At QueryOperationSupplier.java:[lines 54-87]

Possible null pointer dereference in org.apache.phoenix.pherf.workload.mt.tenantoperation.TenantOperationWorkHandler.onEvent(TenantOperationWorkload$TenantOperationEvent) due to return value of called method  Dereferenced at TenantOperationWorkHandler.java:org.apache.phoenix.pherf.workload.mt.tenantoperation.TenantOperationWorkHandler.onEvent(TenantOperationWorkload$TenantOperationEvent) due to return value of called method  Dereferenced at TenantOperationWorkHandler.java:[line 58]

Return value of TenantOperationFactory.getPhoenixUtil() ignored, but method has no side effect  At TenantOperationWorkHandler.java:but method has no side effect  At TenantOperationWorkHandler.java:[line 59]

input must be non-null but is marked as nullable  At UpsertOperationSupplier.java:is marked as nullable  At UpsertOperationSupplier.java:[lines 56-136]

org.apache.phoenix.pherf.workload.mt.tenantoperation.UpsertOperationSupplier$1.apply(TenantOperationInfo) may fail to clean up java.sql.Statement  Obligation to clean up resource created at UpsertOperationSupplier.java:up java.sql.Statement  Obligation to clean up resource created at UpsertOperationSupplier.java:[line 81] is not discharged

input must be non-null but is marked as nullable  At UserDefinedOperationSupplier.java:is marked as nullable  At UserDefinedOperationSupplier.java:[lines 44-46]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/phoenix/pull/1181

JIRA Issue
PHOENIX-6118

Optional Tests
dupname asflicense javac javadoc unit xml compile spotbugs hbaserebuild hbaseanti checkstyle

uname
Linux 3096d137a3a1 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev/phoenix-personality.sh

git revision
master / 2a2d996

Default Java
Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08

checkstyle
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-pherf.txt

whitespace
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/whitespace-eol.txt

whitespace
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/whitespace-tabs.txt

javadoc
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/diff-javadoc-javadoc-phoenix-pherf.txt

spotbugs
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/artifact/yetus-general-check/output/new-spotbugs-phoenix-pherf.html

Test Results
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/testReport/

Max. process+thread count
1570 (vs. ulimit of 30000)

modules
C: phoenix-pherf U: phoenix-pherf

Console output
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1181/1/console

versions
git=2.7.4 maven=3.3.9 spotbugs=4.1.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Most likely I will take a look tomorrow, but I want to know why the 4.x branch includes 58 files changes but master has 55 modified files.

@yanxinyi
The following files are not included in the master, since they had only import changes and was probably taken care by the cherry-pick merges.
SequentialIntegerDataGenerator.java
MultithreadedDiffer.java
WorkloadExecutor.java


Add Reaction class; uncomment Message.reactions and update on events

Related: #121

Updated, resolving small conflicts with #238.

Thanks for the review! Revision pushed. This one updates the reactions field on Message events tracked in MessageListViews.
If you spot ways to make the tests more concise, let me know. 🙂

Then there's a variety of small things most of which apply to the existing tests in that file. For those, I think I'll just go try to work up now a small PR, which may be the most efficient way to describe them.

Sent #258.

Thanks for the review! Revision pushed.

Thanks for the review! Revision pushed, and I made some replies inline above.

Thanks!
All looks good; small nits above. Please go ahead and merge whenever ready.

Thanks for the review!

Please go ahead and merge whenever ready.

Merging, with those changes.


[Feature] Automate Greeting Workflow

Description
I would like to request the addition of an auto-label workflow in this repository to automatically label pull requests with "gssoc-ext" and "hacktoberfest-accepted" during relevant events such as GSSoC (GirlScript Summer of Code) and Hacktoberfest.
Key Features:

Auto-label on PR Creation:

Automatically apply the "gssoc-ext" label for any pull requests submitted as part of GSSoC.
Automatically apply the "hacktoberfest-accepted" label for pull requests during the Hacktoberfest period, or when contributors indicate Hacktoberfest participation.

Workflow Automation:

Use GitHub Actions to create a workflow that triggers upon pull request creation, checking the event or time period, and assigning the appropriate label automatically.

Custom Conditions:

Allow maintainers to manually override the labels if necessary.
Include a check to ensure only valid PRs contributing to the project are labeled with "hacktoberfest-accepted", following Hacktoberfest rules.

Benefits:

Efficiency: Automating this process will save time for maintainers by reducing manual labeling.
Transparency: Ensures that contributors clearly know when their pull request has been accepted for relevant events.
Compliance: Helps in staying aligned with the official rules for Hacktoberfest, where labels like "hacktoberfest-accepted" are required to validate contributions.

Please @Kritika30032002 assign me this issue with labels "gssoc-ext" , "hacktoberfest-accepted" and "level3".
Screenshots
No response
Checklist

[X] I have starred the repository
[X] I have checked the existing issues
[X] I have read the discussion tab thoroughly and got the project idea
[X] I am willing to work on this issue (optional)

Hey @Kritika30032002 title is autolabel workflow


⚠ Mobitopup has degraded performance

In aaf894a, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 6050 ms

Resolved: Mobitopup performance has improved in 851288b after 12 minutes.


Added Editor.NotifyOnce

NotifyOnce can be used to make sure we're not spamming the terminal
with dupe notifications. "End of history" notification is using
this now. Fixes #296

If the user is troubled by the "end of history" message, s/he can just bind Down to a nop in Insert mode: le:binding[Insert][Down]={ }. Note that Down in history mode will still work, no functionality lost.
I believe NotifyOnce will create a grey area between notifications and non-notifications and is prone to abuse. My stand is: if something is notification-worthy, then it should notify the user each time, regardless of whether the last one was the same. If not, then elvish should be silent. So I am closing this PR - sorry :)


Add descriptions to dataset properties, store dataset schemas

This PR adds the capability to add descriptions to Dust apps dataset properties. This is used to produce a DatasetSchema that is stored in DB and that we will be able to leverage to generate inputs to call the app from an custom Assistant.
 After 3 attempts at refactoring it I gave up and added the feature in a minimally invasive way, keeping all the bad patterns that exist there for now :/


Mocking issue

In order to increase my test coverage for the mangopay2 SDK usage, I created a mock of it.
I started by a very simple implementation of the user creation:
// __mocks__/mangopay2-nodejs-sdk.ts
import { default as BaseMangoPay } from 'mangopay2-nodejs-sdk';

class Users extends BaseMangoPay.Users {
  create: BaseMangoPay.Users['create'] = (data) => {
    return new Promise((resolve) => resolve({
      Id: '1337',
    }));
  }
}

export default class MangoPay extends BaseMangoPay {
  Users: Users
}

I have two issues here.
First, typescript is yelling on the Users.create method with the following error:
Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData> & MethodOverload<CreateUserNaturalData, UserNaturalData>'.
  Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData>'.
    Type 'Promise<unknown>' is not assignable to type 'Promise<WithResponse<UserLegalData>>'.
      Type '{}' is missing the following properties from type 'WithResponse<UserLegalData>': statusCode, body, headers

I am quite surprised I have to define the method paremeters and return typing as I already specified which typing to use for this override (BaseMangoPay.Users['create']).
Secondly, my tests are not working with the following error on each one:
    TypeError: Class extends value undefined is not a constructor or null

      1 | import { default as BaseMangoPay } from 'mangopay2-nodejs-sdk';
      2 |
    > 3 | class Users extends BaseMangoPay.Users {
        |                                  ^
      4 |   create: BaseMangoPay.Users['create'] = (data) => {
      5 |     return new Promise((resolve) => resolve({
      6 |       Id: '1337',

      at Object.<anonymous> (__mocks__/mangopay2-nodejs-sdk.ts:3:34)
      at Object.<anonymous> (src/mango.ts:1245:41)

I am quite confused here. How should I manage mocking for that SDK?
Thanks

I finally managed to make the mock working with a correct implementation of the class:
import { default as BaseMangoPay } from 'mangopay2-nodejs-sdk';

export default class MangoPay extends BaseMangoPay {
  constructor(config) {
    super(config);
    this.Users = {
      create: (data) => {
        return new Promise((resolve) => resolve({
          ...data,
          Id: '421337',
        }));
      }
    }
  }
}

However, I still have the typescript typing issue on the Users.create method:
Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData> & MethodOverload<CreateUserNaturalData, UserNaturalData>'.
  Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData>'.
    Type 'Promise<unknown>' is not assignable to type 'Promise<WithResponse<UserLegalData>>'.
      Type '{}' is missing the following properties from type 'WithResponse<UserLegalData>': statusCode, body, headersts(2322)
index.d.ts(3494, 5): The expected type comes from property 'create' which is declared here on type 'Users'

I finally managed to make the mock working with a correct implementation of the class:
import { default as BaseMangoPay } from 'mangopay2-nodejs-sdk';

export default class MangoPay extends BaseMangoPay {
  constructor(config) {
    super(config);
    this.Users = {
      create: (data) => {
        return new Promise((resolve) => resolve({
          ...data,
          Id: '421337',
        }));
      }
    }
  }
}

However, I still have the typescript typing issue on the Users.create method:
Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData> & MethodOverload<CreateUserNaturalData, UserNaturalData>'.
  Type '(data: any) => Promise<unknown>' is not assignable to type 'MethodOverload<CreateUserLegalData, UserLegalData>'.
    Type 'Promise<unknown>' is not assignable to type 'Promise<WithResponse<UserLegalData>>'.
      Type '{}' is missing the following properties from type 'WithResponse<UserLegalData>': statusCode, body, headersts(2322)
index.d.ts(3494, 5): The expected type comes from property 'create' which is declared here on type 'Users'

Hi @soullivaneuh
We're looking into it

Hi @soullivaneuh
We're looking into it

Hi @soullivaneuh
You should try to mock the creation of a LEGAL or NATURAL user. Using just user it won't work.
Mircea


Fix various typos

Fix various typos.

@alexandair : Thanks for your contribution! The author, @ggailey777, has been notified to review your proposed change.

Thanks @alexandair! I have some other work ongoing in this topic, so I will pull your commit into the private repo and incorporate your suggestions.

@ggailey777 Do you want this PR closed? Can you link to the update so Aleksandar can get recognition for making a contribution?

Yes, #please-close. Sometimes GitHub closes the PR after the commit makes it back into the repo, but I guess not in this case. Note, however, that credit has been granted 😁


automatically bump HG_LOG_SUBSYS when HG_LOG_LEVEL is set

Is your feature request related to a problem? Please describe.
See https://github.com/mochi-hpc/mochi-thallium/issues/7#issuecomment-1379100285.  The log subsys is not implicitly changed in response to the log level when set via env variable, but it is when set by library API, if I understand correctly.
Describe the solution you'd like
It would probably be better if they were consistent, so that users who want to see non-fatal error messages can get there with one environment variable.
Describe alternatives you've considered
We are going to address this in Margo via https://github.com/mochi-hpc/mochi-margo/issues/233 (we would prefer a higher default log level for our use case anyway so we may as well take explicit control over both the log level and log subsys for our purposes), just noting that maybe the inconsistency between the env variables and and programmatic API might be a good thing to fix independently at the Mercury level for non-Margo users.

I fixed that just now, setting HG_LOG_LEVEL without setting HG_LOG_SUBSYS will be equivalent to setting HG_LOG_SUBSYS=hg_all (meaning all subsys enabled)


[Suggestion] Remove SDKRef.md

Note: This repository is ONLY used to solve issues related to DOCS.
For other issues, please move to other repositories.

Is there anything that's missing or inappropriate in the docs? Please describe.
SDKRef.md is unnecessary.
Describe your suggestion
Remove SDKRef.md.

Fixed.


Android issue when tintColor is added

I've read the issues and I believe this is not a duplicate.
When using android and specifying a tintColor, the following issue arises.

I believe it's an issue over the parseColor function.

Hello!
It is a duplicate. Related issue: #29
I opened a PR. You can check the comment for a workaround.


Operation.composite/3 doesn't call to_nif_term/2 for blend modes

Vix.Vips.Operation.composite/3 takes a list of blend modes as the second argument. However while the type is annotated as [integer()] (which is correct given it is declared as int *mode in libvips), this would normally be expected to be [Vix.Vips.Enum.VipsBlendMode.t()] to be consistent with other functions.
I appreciate that these functions are generated at compile time based upon introspecting libvips. Perhaps therefore there is no "fix" for this. But maybe a documentation entry (also difficult given I know the docs are auto generated) to note the need to call Vix.Vips.Enun.VipsBlendMode.to_nif_term/2 would be helpful?

Yes, I'm aware of this. Types we get from libvips is flat (VipsArrayInt, VipsArrayImage etc) so this is something we have to do at our end. I think we can have some level of abstraction in place to support casting. I'll look into this. to_nif_term is meant to be for internal use, so I  want to avoid exposing those to the user.
Thanks again for the contribution!


SDK: Remove columns roles that are not usable by SDK

Column roles like "Wait Duration" don't really apply to all plugins

1.0:

Remove from SDK
Update Plugins

Prune columns - update serialization/deserialization of table configs. T-shirt size: M


404 on ember install

ember install ember-simple-auth-token
version: 1.13.13
Registry returned 404 for GET on https://registry.npmjs.org/ember-simple-auth-token

Hi, try running ember-cli-simple-auth-token. This will install ember-cli-simple-auth on the 0.7.1 version. This version is not working with ember-simple-auth version 1.0.0 so you should use 0.8.0.

Just published v1.0.0 https://www.npmjs.com/package/ember-simple-auth-token


Show network name in exception message

Description

Display network name in exception message instead of network object

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[X] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):
How Has This Been Tested?

Thanks for the PR @ravening.
I am +1 on adding the network's name on exception. I did not add a looks good for this PR in respect to Daan's comment.

@GabrielBrascher Daan was mentioning about networkmodel retuning a null rather than network name being null. So i think this should be fine and no need for extra if check

@ravening can you fix the conflict?

@ravening can you fix the conflict?

@rhtyd done

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2296

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-3097)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31757 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4396-t3097-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_04_rvpc_privategw_static_routes
Failure
869.32
test_privategw_acl.py

@rhtyd @ravening
build failed due to a missing ")" at the end of line.

                 // * verify that there are no duplicates
                 if (hostNames.contains(hostName)) {
                     throw new InvalidParameterValueException("The vm with hostName " + hostName + " already exists in the network domain: " + ntwkDomain.getKey() + "; network="
-                            + (_networkModel.getNetwork(ntwkId) != null) ? _networkModel.getNetwork(ntwkId).getName()) : "<unknown>";
+                            + (_networkModel.getNetwork(ntwkId) != null) ? _networkModel.getNetwork(ntwkId).getName() : "<unknown>");
                 }
             }
         }

good it has been fixed by @rhtyd


Add gsys.biz to the list of 11ty sites.

Yet another 11ty generated site to consider for inclusion in the speedlify list.
It’s AMP based, hope you won’t hold that against it!
Many thanks for producing this great tool.

I’m seeing infinite redirects here on your URL!

Oh no! That's not good!
As is always the case, it works here :-(
To try and help me debug this horrible situation could I have a copy of the HAR file?
I'll just check... the edit your ending up with has a URI of http://www.gsys.biz/  ?

Looks like I’m getting redirected to /.well-known/captcha/

Thanks for the response... it's really helpful.
The /.well-known/captcha/ endpoint was news to me.
After some digging I found an article at https://www.siteground.com/blog/new-anti-bot-ai/ which proved to be useful. It appears the ineffable AI has decided you are a bot.
I have contacted SiteGround support and asked them to disable the anti-bot-ai which they say has now been done.
Hopefully you (and God knows how many others) can now access the site.

Looks good, thanks!


Insufficient documentation for warning C4770

The only documentation for warning C4770 is the re-statement of the message on this page.
Can I request some more details about that warning, since I've started seeing it in one of our projects and don't know what it means or what the implications are.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 1b453bef-7a9d-b904-16cd-1c575d8e34f9
Version Independent ID: 5e564e6f-2ac2-d898-690f-bcdd4502ed5d
Content: Compiler warnings C4600 Through C4799
Content Source: docs/error-messages/compiler-warnings/compiler-warnings-c4600-through-c4799.md
Product: visual-cpp
Technology: cpp-diagnostics
GitHub Login: @corob-msft
Microsoft Alias: corob

@danieljennings
You'll see this warning if you cast or alias an enum value as an integral type to use as an index, but don't validate that the resulting value is non-negative and possibly also that it's <= the maximum enum value of that type. (This is what "partially validated" means in this context). This warning isn't new; it's been in the product since VS 2013, and the code that checks for the warning condition hasn't changed in at least five years. I'd be very surprised if it started appearing in code you haven't changed, unless you're upgrading from an older version than VS 2013, or enabling warnings that weren't enabled before.

Thank you! I will dig into this more tomorrow but thanks to your note re: timeframe I've found that this is occurring in a project that didn't have LTCG enabled before but had it enabled recently due to a lower-level project change. Not the sort of warning I would have predicted to come from LTCG, but now I have an explanation for how it started happening and for what it means, so thank you, @corob-msft .

I realize that this is an optional warning, but I wanted to close the loop here with why I was confused before I head out for trick or treating :) In our case we're not using it as an "index" in the strict sense of the word but we are turning two enums into an integral value to use for other things. I can start to understand the warning if the intention is along the lines of "you're using an int derived from an enum and only range-checking one piece that went into that int", but since we aren't actually doing any "indexing" I was confused at first. (Also I initially didn't expect compiler-looking warnings to pop up after enabling /GL but that's less surprising the more I think about it, too.)
Here's a trimmed down repro case for what we hit, in case it's either surprising to your team that it produces the warning or somehow otherwise useful to see where it hits in real-world scenarios. In our case here we've just disabled the warning for the project in question, so it's no big deal even if nothing changes here.
// Compiled with: /w14770 /GL

enum MyEnum_t
{
	MyEnum_A,
	MyEnum_B,
	MyEnum_C,
};

int main( const char *[], int argc )
{
    // Obviously dummy not-known-at-compile/link-time values
    const MyEnum_t nFirst = (MyEnum_t)argc;
    const MyEnum_t nSecond = (MyEnum_t)( argc + 1 );

    const int nTotal = ( nFirst + nSecond ); // <---- this line triggers C4770
    if ( nFirst <= MyEnum_B )
    {
        return nTotal;
    }

    return 0;
}

Again, I appreciate your (especially quick!) response here. Thank you, and have a great Halloween :)

Content is available at Compiler Warning (level 4) C4770.

Thank you!


add and remove metaData plots

Try using (https://boxicons.com/usage)
Plus icon in plotRow header
Launches a modal (https://getbootstrap.com/docs/5.0/components/modal/) - this is a form to populate the necessary data and layout objects. Also needs bar / histogram / scatter dropdown.
Then remove plot icon in plotTitle.

Hi Graham,
I've been working on the plot removal/addition. When removing a plot there are some side effects due to the D3 approach to plot making/updating/removing.
Issues are:

Plots may paste their graphics over other graphics. E.g.: in a sequence of bar, bar, scatter, histogram (from test/session.json), if a bar chart is removed the new scatterplot will have a bar chart in the background.
Histograms may throw an error when being updated.  E.g.: in a sequence of bar, bar, scatter, histogram (from test/session.json), if any bar/scatter chart is removed the histogram will break because it cannot find "xDomMin" and "xDomMax", and hterefore cannot construct its bins.
Because all plots make use of makeNewPlot, where the remove button was implemented, all plots can be removed - including the contour small multiples!

The issues arise because of how our D3 approach pairs up the data to the HTML elements, and arises only only when changing the plot/plot row data.
When a plot in the middle of the plot row is removed, its plotData is removed from dbslice.session, and `update' is called to refresh the session. Within the changed plot row, D3 will pair the plotData objects to <div.plot> in sequence, meaning that the plotData after the removed plot move to the preceding <div.plot> and its content. In short: the desired plotData is removed, but the HTML removed is that of the last plot. This then causes the overlap in the graphics. In contour plots the data moves between small multiples, but the title is not updated. The issue with the histogram occurs as the histogram saves the "xDomMin" and "xDomMax" as an attribute of an   element, which is not available after the plot migrates to another <div.plot>.
To fix the graphics overlapping problem the plots have to be cleared everytime the plots, and possibly plot rows, are removed. For histogram issues the histogram would have to be changed to not rely on attributes of the HTML or SVG elements. Furthermore, when plots are migrating between <div.plot> the remove plot functionality has to be refreshed to get access to the correct plotData.

A more elegant approach may be to restructure the plots and the plot adding/removing approach. This could be useful also when implementing the ability to allow the user to change the plotted variables (I already wrote some modules that allow this).
in update.js:
function update( elementId = dbsliceData.elementId, session = dbsliceData.session ){
...
let plotRows = dbslice.session.plotRows;
// Filter out plots that need to be removed.
plotRows = plotRows.filter(p=>{
if(!p.active){
p.remove( ... )
}
return p.active
})
// Make new plots and update exsiting ones.
dbslice.session.plotRows.forEach(p=>{
if(p.node){
p.update( ... )
} else {
p.make( ... )
container.appendChild( p.node )
} // if
})
...
} // update
Plot rows can be introduced as a separate object:
const plotRow = {
make: function(){...},
update: function(){...},
remove: function(){...}
}
Plots can be expanded to take control of their HTML and other data, which is useful also when obtaining plotConfigs, as different plots can respond with different types of plot configs.
const cfD3Plot = {
node : undefined,

xDomMin : undefined,
xDomMax : undefined,

make : function(){
	...
	-- var container = d3.select(element);

	...

	-- var svg = container.append("svg")
    --  .attr("width", svgWidth)
    --  .attr("height", svgHeight)

	++ this.node = d3.create("svg")
    ++  .attr("width", svgWidth)
    ++  .attr("height", svgHeight)
	++  .node()

	// Within an object "this" is the parent object.

	...

	// Keep track of plot specific values
	var xDomMax = itemExtent[1] + 0.05*itemRange;
    -- plotArea.attr( "xDomMax", xDomMax);
	++ this.xDomMax = xDomMax;

	...

},

update : function(){ ... }

} // cfD3Plot


Fix potential infinite loop

On a headless linux build we ran into an issue where the server would just lock up for no reason
After a lot of fiddling about to figure out how to debug it (attach gbd + http://www.mono-project.com/docs/debug+profile/debug/#debugging-with-gdb ), we noticed it got stuck in UMAGeneratorCoroutine.OptimizeAtlas
It seems that the usedArea can end up being 0 on either axis in some cases, which will cause it to loop for a long time (endlessly?)
This is a quick and dirty fix to bypass the issue, I wasn't exactly sure how to handle this case properly so i just left GeneratedMaterial.cropResolution at the default value.
Can this be set to 0 or some other small value? Does it matter? (Havent had the time to dig through the code myself)
Not super sure on the 0 (Mathf.Abs(...) > 0.0001 ) check either, it should be fine?
I'd be more than happy to amend the patch if you could give me some pointers - or feel free to close & implement a better fix yourself

Hey, good catch.  I'm thinking if either x axis  or y axis is zero then both of those loops should be skipped entirely.  @Jaimi @ecurtz do you have any input?
I think ultimately it'd be better to check for headless mode and skip large parts of UMA, like the atlas generation.

Yeah, I agree. I'll merge this


Show spell content as a toggle

https://app.roll20.net/forum/permalink/3264216/

Done as part of 2.6.2


feat: add stats doc on migration

The tests are not passing because of a TZ mismatch with snapshots. merging


Issue 1513: Unit test for RemoteRepositoryAlivenessCacheManager

Added initial test using a mocked remote repository.
The putShouldChangeCachedValueImmediately parameterized test fails in two cases, which merits further investigation. This might be a problem with the Hazelcast Cache not updating values immediately.
Eviction strategy for isAliveShouldReturnTrueWhenRemoteRepositoryCachedValueIsExpired still to be done.

Pull Request Description
This pull request is a draft. I'd appreciate a bit of feedback on this as-is.
I would like someone to take a look at the parameterized test that currently fails. It seems to me this should be passing, so perhaps there's an issue with the Hazelcast cache.
A bit of help on how to forcibly expire a cache item would be appreciated. I could forcibly just evict the cache entry by using Cache.evict, but could be a better way.
Acceptance Test

[ ] Building the code with mvn clean install -Dintegration.tests still works.
[ ] Running mvn spring-boot:run in the strongbox-web-core still starts up the application correctly.
[ ] Building the code and running the strongbox-distribution from a zip or tar.gz still works.
[ ] The tests in the strongbox-web-integration-tests still run properly.

Questions

Does this pull request break backward compatibility?

[ ] Yes
[X] No

Does this pull request require other pull requests to be merged first?

[ ] Yes, please see #...
[X] No

Does this require an update of the documentation?

[ ] Yes, please see strongbox/strongbox-docs#{PR_NUMBER}
[X] No

@mwvdev ,
Thanks for your pull request!
Would you mind adding some more details, explaining the error and adding the exception you're getting? Thanks! :)
@sbespalov , @fuss86 : Could you please have a look and review? Thanks! :)

Actually, seems I was a bit too fast on the trigger. I've found a few mistakes that I'm going to be looking into fixing. The isAliveShouldReturnFalseWhenRemoteRepositoryCachedValueIsFalse is not asserting what it claims to be... right now it's just asserting true, which indicates there's probably a more general problem me assigning values to the cache.
Having looked at it, I'm actually not able to assign any values to the cache, it seems to just rely on the fact that isAlive returns true for when the cache either contains a null or true value.

@fuss86 I'm having troubles getting anything to actually be persisted in the cache. I've tried debugging through the various steps but I'm not sure why put operations aren't persisted in the cache. Could you perhaps have a look at the isAliveShouldReturnFalseWhenRemoteRepositoryCachedValueIsFalse unit test to see if you can spot any issues?

@mwvdev thanks for the PR!
can you please proceed with comments?

@sbespalov Thanks for the feedback! I'll start working my way through the comments.


package-info.class cause "org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar" failure since 2.59.0

org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar raise below ERROR message.
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar (default-cli) on project ****-*******-******: Can not execute Findbugs: java.lang.AssertionError: Can't find ClassInfo for com\gargoylesoftware\htmlunit\package-info

Please switch to 2.60.0 this is (hopefully) already fixed.

There are  package-info.class in htmlunit 2.60.0

Strange - my guess was this is related to #457.

From the Mavan PR https://github.com/apache/maven-compiler-plugin/pull/88

This option is turned on by default because the generated class files are valid, so they should not cause any problem.

Can you please verify that the version 2.60.0 still causes problems with sonarsource.scanner - yes i know that the files are still available in 2.60.0 but they are now correct java files and should not harm.

From the Mavan PR apache/maven-compiler-plugin#88

This option is turned on by default because the generated class files are valid, so they should not cause any problem.

Can you please verify that the version 2.60.0 still causes problems with sonarsource.scanner - yes i know that the files are still available in 2.60.0 but they are now correct java files and should not harm.

It is verified that sonarsource.scanner could run property for htmlunit 2.60.0.

@asdf913 means i can close this?

@asdf913 means i can close this?
Yes

Thanks for the feedback, enjoy using HtmlUnit


[Extensions] Add AOT compatibility attributes to Microsoft.Extensions.Azure

Annotate the Microsoft.Extensions.Azure library for trimming/AOT compatibility.
Also add a few suppressions to Azure.Identity's EventSource that showed up when I was using Microsoft.Extensions.Azure in a real-world app.
Continues work started in #35572.
cc @m-redding

API change check
APIView has identified API level changes in this PR and created following API reviews.
Microsoft.Extensions.Azure


Application side policy implementation

Adds a simple application side policy implementation.
During bootstrap the application will define a single PolicyMap with the roles and permissions for the default policy and optionally any other policies desired.
Upon completion of successful User Authentication, the application will receive a UserDetails to be passed into the GetUserPolicy method of the PolicyMap. This will return a UserPolicy that can be used to test if the current user has any role or permission via the HasRole and HasPermission methods on the UserPolicy object.
See the can_create_policy_map test for an example of usage
note: the application can implement the PolicyMap and mock UserDetail records for developer policy testing without requiring connection to the authentication server.
pending

[ ] Export policy method on the policy map for writing a json export file
[ ] Additional tests

From the example in the tests, I really like the usage of the PolicyMap in combination with UserPolicy. It looks really straightforward for devs to use.
One question: for the user with multiple roles, e.g., one who has both the "user" and "admin" roles from the can_create_policy_map test, how do we ask the question "does the user have permission to do X when operating in the 'user' role?" In other words, how do we enforce a sudo-like privilege escalation scheme whereby we say "Yes, you can do that, but it requires the more privileged role so we need confirmation"? It looks to me like the permissions in UserPolicy have no memory of which role they originated from. I think we need something in UserPolicy like bool HasPermission(Permission, Role), which implies that UserPolicy needs to know which role allows a given permission.

It might make sense to have the role be optional in HasPermission, i.e., without specifying a role we're just asking "can the user do X" regardless of role, while specifying a role refines the query to "can the user do X when using role Y?"

Interactive rebase for Merge


How to stop http task(rest call) execution while workflow start

How to stop http task(rest call) execution while workflow start.
My workflow definition have only one http task

Closing this issue as there is no activity for a while. Please feel free to reopen if you still have questions or concerns.


Notification: "NVD3D10: CPU cyclestats are disabled on client virtualization"

I followed the instructions on the page titled "Tensorflow with DirectML".  I installed WSL 2 yesterday and updated the Linux kernel to <IP_ADDRESS>-microsoft-standard-WSL2 today.  The Windows 10 build is 21359.1.  I followed the instructions to install Miniconda, Python, and then Tensorflow and DirectML.  I started an interactive session of Python and entered the code to add two tensors.  I got the expected output, but I also got 3 lines of the subject notification.  Other system information includes:
Processor: AMD Ryzen Threadripper 3960x
GPU: GEForce RTX 2080 Ti
NVIDIA Graphics Driver: 470.25

That's a normal thing to see when running the preview NVIDIA WSL driver.

Thank you for the very fast response.
From: Justin Stoecker @.>
Sent: Wednesday, April 21, 2021 5:05 AM
To: microsoft/DirectML @.>
Cc: markwny @.>; Author @.>
Subject: Re: [microsoft/DirectML] Notification: "NVD3D10: CPU cyclestats are disabled on client virtualization" (#107)
That's a normal thing to see when running the preview NVIDIA WSL driver.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FDirectML%2Fissues%2F107%23issuecomment-823904363&data=04|01||53cc5566f3894efea4f408d904a47df1|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637545926791643003|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=7N%2BqylVCZUIGOnxpnrJBUByN1UY2px5gn0ttr6J7lJU%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALA4CRJCLBYDPOWSTAST6P3TJ2IKLANCNFSM43JSH35Q&data=04|01||53cc5566f3894efea4f408d904a47df1|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637545926791652997|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=RBrGeoatk%2FyFWK0Zn0ZduBX9LzpS%2FXldJL%2F7rsh%2F%2Bko%3D&reserved=0.

Closing for now. If you have concerns or questions please feel free to reopen!

so, why do we meet it, and how to solve it?


[Enhancement] Support replication from non-TDE cluster to TDE cluster

Why I'm doing:
Currently, directly downloaded file(like in replication and clone) are not encrypted, so  eplication from non-TDE cluster to TDE cluster is not supported.
What I'm doing:
Support replication from non-TDE cluster to TDE cluster
Fixes #49384
What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

[ ] This is a backport pr

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.3
[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5

@Mergify rebase


Fix self in Module#instance_eval

irb(main):001:1* class Foo
irb(main):002:1*   def foo; end
irb(main):003:1*   def self.bar; end
irb(main):004:0> end
=> :bar
irb(main):005:0> Foo.instance_eval { alias foo2 foo }
(irb):5:in `block in <main>': undefined method `foo' for class `#<Class:Foo>' (NameError)
	from (irb):5:in `instance_eval'
	from (irb):5:in `<main>'
	from /usr/local/Cellar/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/irb-1.3.5/exe/irb:11:in `<top (required)>'
	from /usr/local/opt/ruby/bin/irb:23:in `load'
	from /usr/local/opt/ruby/bin/irb:23:in `<main>'
irb(main):006:0> Foo.instance_eval { alias_method :foo2, :foo }
=> :foo2
irb(main):007:0> Foo.instance_eval { alias_method :bar2, :bar }
(irb):7:in `alias_method': undefined method `bar' for class `Foo' (NameError)
	from (irb):7:in `block in <main>'
	from (irb):7:in `instance_eval'
	from (irb):7:in `<main>'
	from /usr/local/Cellar/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/irb-1.3.5/exe/irb:11:in `<top (required)>'
	from /usr/local/opt/ruby/bin/irb:23:in `load'
	from /usr/local/opt/ruby/bin/irb:23:in `<main>'
irb(main):008:0> Foo.instance_eval { alias bar2 bar }
=> nil


ETA on next release

👋 sorry to bug about this in an issue tracker, but I was curious about an ETA for the next release? The PR I recently made being in a hex release is the only thing holding up the 0.1.0 release of ash_sqlite :).

SOON(tm), working on it now

Awesome, thank you :) don't rush on my account, just wanted to keep my finger on the pulse.

I was cutting v0.15.0 for exqlite and noticed I needed to also bump this too because your fix was in there.


Execution hangs when accessing a object member function in select clause

Description:
forever {
        from inputStream
        select std.getMarks() as marksOfStudent
        => (TeacherOutput[] emp) {
            foreach var e in emp {
                outputStream.publish(e);
            }
        }
    }

type Student object {
   string abcd = "Lahiru";
    int marks = 45;

    function getMarks() returns int {
        return self.marks;
    }
};

This code hangs in execution infinitely.
Suggested Assignees (optional):
@mohanvive , @gimantha , @grainier

This issue is fixed in  #13761


Change resource access from HTTP to HTTPS

Fixes #467
This PR modifies a JS script we're using right now to fetch emoji images to add support for HTTP in order to avoid security problems. Right now it seems there are no other conflictive resources being loaded as HTTP. Could you please review, @dialelo? Thanks!! :)

LGTM :+1:


I get an error when running code for chapter 3.

Got an attribute error: The 'Book' object has no attribute 'review_set' Kindly help.

Hi Clarion,
Could you please give me a few details so that I can figure out the issue better?

Django version you're using (3.0, 3.1, or 4.0) to run the code on
Python version you're using to run the code on
Operating system
The page number on which you got the error

Looking forward to hearing from you!

I'm getting the same error in Chapter 3.
Django version 4.2.8
Python version 3.9
OS: Linux Penguin 6.1.55 (on a Google Pixelbook)
Pages 136-138. The code example on p. 136 is missing the new part of the code (the book_list function), so I had to pull in from the Github repo
This is the offending bit of code:
def book_list(request):
books = Book.objects.all()
book_list = []
for book in books:
reviews = book.review_set.all()
if reviews:
book_rating = average_rating([review.rating for review in reviews])
number_of_reviews = len(reviews)
else:
book_rating = None
number_of_reviews = 0
book_list.append(
{
"book": book,
"book_rating": book_rating,
"number_of_reviews": number_of_reviews,
}
)
context = {"book_list": book_list}
return render(request, "reviews/book_list.html", context)


Cannot install dcos spark cli with DC/OS 1.8

What version of DC/OS + DC/OS CLI are you using (dcos --version)?
DC/OS 1.8 using AWS CloudFormation template from https://dcos.io/docs/1.8/administration/installing/cloud/aws/
DC/OS CLI:
dcoscli.version=0.4.10
dcos.version=N/A
dcos.commit=N/A
dcos.bootstrap-id=N/A
What operating system and version are you using?
Ubuntu 16.04.01 LTS
What did you do?
dcos --log-level=DEBUG package install spark --cli

What did you expect to see?
Spark on DC/OS was installed via the Web UI from the universe. I expected that I can install the DC/OS spark CLI afterwards, which used to work with DC/OS 1.7 and the previous version of the DC/OS CLI.
What did you see instead?
mstuder@falcon:~/dcos$ dcos --log-level=DEBUG package install spark --cli
MainThread: 2016-08-08 21:01:31,441 dcos/http.py:_request:87 - Sending HTTP ['get'] to ['http://dcos-elasticloadba-gs37t0u7mxpm-42049901.eu-central-1.elb.amazonaws.com/capabilities']: {'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1', 'Accept': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1'}
MainThread: 2016-08-08 21:01:31,448 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:213 - Starting new HTTP connection (1): dcos-elasticloadba-gs37t0u7mxpm-42049901.eu-central-1.elb.amazonaws.com
MainThread: 2016-08-08 21:01:31,599 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:395 - "GET /capabilities HTTP/1.1" 200 82
MainThread: 2016-08-08 21:01:31,601 dcos/http.py:_request:117 - Received HTTP response [200]: {'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1', 'Server': 'openresty/<IP_ADDRESS>', 'Connection': 'keep-alive', 'Content-Length': '82', 'Date': 'Mon, 08 Aug 2016 19:01:31 GMT'}
MainThread: 2016-08-08 21:01:31,601 dcos/util.py:timer:529 - duration: dcos.http._request: 0.16s
MainThread: 2016-08-08 21:01:31,602 dcos/http.py:_request:87 - Sending HTTP ['post'] to ['http://dcos-elasticloadba-gs37t0u7mxpm-42049901.eu-central-1.elb.amazonaws.com/package/describe']: {'Content-Type': 'application/vnd.dcos.package.describe-request+json;charset=utf-8;version=v1', 'Accept': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2'}
MainThread: 2016-08-08 21:01:31,604 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:213 - Starting new HTTP connection (1): dcos-elasticloadba-gs37t0u7mxpm-42049901.eu-central-1.elb.amazonaws.com
MainThread: 2016-08-08 21:01:31,755 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:395 - "POST /package/describe HTTP/1.1" 200 13068
MainThread: 2016-08-08 21:01:31,815 dcos/http.py:_request:117 - Received HTTP response [200]: {'Content-Type': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2', 'Server': 'openresty/<IP_ADDRESS>', 'Connection': 'keep-alive', 'Content-Length': '13068', 'Date': 'Mon, 08 Aug 2016 19:01:31 GMT'}
MainThread: 2016-08-08 21:01:31,815 dcos/util.py:timer:529 - duration: dcos.http._request: 0.21s
Installing CLI subcommand for package [spark] version [1.0.1-2.0.0]
MainThread: 2016-08-08 21:01:31,816 dcos/util.py:ensure_dir_exists:129 - Creating directory: '/home/mstuder/.dcos/subcommands/spark'
MainThread: 2016-08-08 21:01:31,817 dcos/subcommand.py:_execute_command:571 - Calling: ['/usr/local/bin/virtualenv', '--version']
MainThread: 2016-08-08 21:01:31,859 dcos/subcommand.py:_execute_command:584 - Command script's stdout: b'15.0.2\n'
MainThread: 2016-08-08 21:01:31,860 dcos/subcommand.py:_execute_command:585 - Command script's stderr: b''
MainThread: 2016-08-08 21:01:31,860 dcos/subcommand.py:_execute_command:571 - Calling: ['/usr/local/bin/virtualenv', '/home/mstuder/.dcos/subcommands/spark/env']
MainThread: 2016-08-08 21:01:32,658 dcos/subcommand.py:_execute_command:581 - Command script's stdout: b'New python executable in /home/mstuder/.dcos/subcommands/spark/env/bin/python\nInstalling setuptools, pip, wheel...\n  Complete output from command /home/mstuder/.dcos/...spark/env/bin/python - setuptools pip wheel:\n  Traceback (most recent call last):\n  File "<stdin>", line 7, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/__init__.py", line 16, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/vcs/subversion.py", line 9, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/index.py", line 30, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/wheel.py", line 39, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/_vendor/distlib/scripts.py", line 14, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/_vendor/distlib/compat.py", line 31, in <module>\nImportError: cannot import name HTTPSHandler\n----------------------------------------\n...Installing setuptools, pip, wheel...done.\n'
MainThread: 2016-08-08 21:01:32,658 dcos/subcommand.py:_execute_command:582 - Command script's stderr: b'Traceback (most recent call last):\n  File "/usr/local/bin/virtualenv", line 11, in <module>\n    sys.exit(main())\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 711, in main\n    symlink=options.symlink)\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 944, in create_environment\n    download=download,\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 900, in install_wheel\n    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 795, in call_subprocess\n    % (cmd_desc, proc.returncode))\nOSError: Command /home/mstuder/.dcos/...spark/env/bin/python - setuptools pip wheel failed with error code 1\n'
Error installing 'spark' package.
Run with `dcos --log-level=ERROR` to see the full output.

As suggested by @mgummelt  this issue was moved here from https://github.com/mesosphere/dcos-spark/issues/35

@martinstuder Can you run dcos --log-level=debug --version against and include the dcos... version information? The version of DC/OS you are using is important.
@tamarrow @mgummelt, I see from the logs that we are trying to install the spark CLI using pip. Shouldn't we be trying to install using the native CLI packaging?

@martinstuder Can you run dcos --log-level=debug --version against and include the dcos... version information? The version of DC/OS you are using is important.
@tamarrow @mgummelt, I see from the logs that we are trying to install the spark CLI using pip. Shouldn't we be trying to install using the native CLI packaging?

@jsancio I was running the above against DC/OS 1.8 but re-ran the same against the newer DC/OS 1.8.1 AWS CloudFormation template:
dcos --log-level=DEBUG --version package install spark --cli
MainThread: 2016-08-15 20:02:40,724 dcos/http.py:_request:87 - Sending HTTP ['get'] to ['http://dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com/dcos-metadata/dcos-version.json']: {'Accept': 'application/json'}
MainThread: 2016-08-15 20:02:40,729 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:213 - Starting new HTTP connection (1): dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com
MainThread: 2016-08-15 20:02:40,876 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:395 - "GET /dcos-metadata/dcos-version.json HTTP/1.1" 200 154
MainThread: 2016-08-15 20:02:40,877 dcos/http.py:_request:117 - Received HTTP response [200]: {'Accept-Ranges': 'bytes', 'Server': 'openresty/<IP_ADDRESS>', 'Date': 'Mon, 15 Aug 2016 18:02:40 GMT', 'Content-Length': '154', 'Last-Modified': 'Thu, 11 Aug 2016 01:33:48 GMT', 'ETag': '"57abd5fc-9a"', 'Content-Type': 'application/json', 'Connection': 'keep-alive'}
MainThread: 2016-08-15 20:02:40,877 dcos/util.py:timer:529 - duration: dcos.http._request: 0.15s
dcoscli.version=0.4.10
dcos.version=1.8.1
dcos.commit=c1915a9f9f02caf7e34022eaea04f15ff853bd0e
dcos.bootstrap-id=aab1123cef197db2d94bd8ba024c63a99b876958

dcos --log-level=debug package install spark --cli
MainThread: 2016-08-15 20:04:29,713 dcos/http.py:_request:87 - Sending HTTP ['get'] to ['http://dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com/capabilities']: {'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1', 'Accept': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1'}
MainThread: 2016-08-15 20:04:29,718 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:213 - Starting new HTTP connection (1): dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com
MainThread: 2016-08-15 20:04:29,853 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:395 - "GET /capabilities HTTP/1.1" 200 103
MainThread: 2016-08-15 20:04:29,854 dcos/http.py:_request:117 - Received HTTP response [200]: {'Date': 'Mon, 15 Aug 2016 18:04:29 GMT', 'Connection': 'keep-alive', 'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1', 'Server': 'openresty/<IP_ADDRESS>', 'Content-Length': '103'}
MainThread: 2016-08-15 20:04:29,854 dcos/util.py:timer:529 - duration: dcos.http._request: 0.14s
MainThread: 2016-08-15 20:04:29,855 dcos/http.py:_request:87 - Sending HTTP ['post'] to ['http://dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com/package/describe']: {'Content-Type': 'application/vnd.dcos.package.describe-request+json;charset=utf-8;version=v1', 'Accept': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2'}
MainThread: 2016-08-15 20:04:29,856 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:213 - Starting new HTTP connection (1): dcos-elasticloadba-17q7kqx0oopde-1788100312.eu-central-1.elb.amazonaws.com
MainThread: 2016-08-15 20:04:29,985 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:395 - "POST /package/describe HTTP/1.1" 200 13003
MainThread: 2016-08-15 20:04:29,986 dcos/http.py:_request:117 - Received HTTP response [200]: {'Date': 'Mon, 15 Aug 2016 18:04:29 GMT', 'Connection': 'keep-alive', 'Content-Type': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2', 'Server': 'openresty/<IP_ADDRESS>', 'Content-Length': '13003'}
MainThread: 2016-08-15 20:04:29,986 dcos/util.py:timer:529 - duration: dcos.http._request: 0.13s
Installing CLI subcommand for package [spark] version [1.0.1-2.0.0]
MainThread: 2016-08-15 20:04:29,987 dcos/util.py:ensure_dir_exists:129 - Creating directory: '/root/.dcos/subcommands/spark'
MainThread: 2016-08-15 20:04:29,988 dcos/subcommand.py:_execute_command:571 - Calling: ['/usr/local/bin/virtualenv', '--version']
MainThread: 2016-08-15 20:04:30,031 dcos/subcommand.py:_execute_command:584 - Command script's stdout: b'15.0.2\n'
MainThread: 2016-08-15 20:04:30,032 dcos/subcommand.py:_execute_command:585 - Command script's stderr: b''
MainThread: 2016-08-15 20:04:30,032 dcos/subcommand.py:_execute_command:571 - Calling: ['/usr/local/bin/virtualenv', '/root/.dcos/subcommands/spark/env']
MainThread: 2016-08-15 20:04:30,713 dcos/subcommand.py:_execute_command:581 - Command script's stdout: b'New python executable in /root/.dcos/subcommands/spark/env/bin/python\nInstalling setuptools, pip, wheel...\n  Complete output from command /root/.dcos/subcommands/spark/env/bin/python - setuptools pip wheel:\n  Traceback (most recent call last):\n  File "<stdin>", line 7, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/__init__.py", line 16, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/vcs/subversion.py", line 9, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/index.py", line 30, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/wheel.py", line 39, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/_vendor/distlib/scripts.py", line 14, in <module>\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-8.1.2-py2.py3-none-any.whl/pip/_vendor/distlib/compat.py", line 31, in <module>\nImportError: cannot import name HTTPSHandler\n----------------------------------------\n...Installing setuptools, pip, wheel...done.\n'
MainThread: 2016-08-15 20:04:30,713 dcos/subcommand.py:_execute_command:582 - Command script's stderr: b'Traceback (most recent call last):\n  File "/usr/local/bin/virtualenv", line 11, in <module>\n    sys.exit(main())\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 711, in main\n    symlink=options.symlink)\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 944, in create_environment\n    download=download,\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 900, in install_wheel\n    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)\n  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 795, in call_subprocess\n    % (cmd_desc, proc.returncode))\nOSError: Command /root/.dcos/subcommands/spark/env/bin/python - setuptools pip wheel failed with error code 1\n'
Error installing 'spark' package.
Run with `dcos --log-level=ERROR` to see the full output.

I actually get the same problem when trying to install the kafka subcommand. The cassandra one works though.

Same problem when I try to install spark cli.

Same here with Azure and the subcommands for Spark / Kafka but for me the cassandra command is also broken.
dcoscli.version=0.4.10
dcos.version=1.8.3


added discountRules intro; deleted discounts references

@guyofeck can you please approve? I deleted the obsolete "Discounts" for the correctly named "Discount Rules"


Input.Toggle valueOn - valueOff makes no sense

Input Toggle enables the user to specify if the valueOn property is either a True or False value. The issue with this is that the valueOff can be set to exactly the same value.
As a Toggle is being presented as a CheckBox, these two properties should be removed and the traditional IsChecked property to denote that it is on or off

Hi there, the original intention for valueOn and valueOff was used with the Action.Http action (which we've since removed from v1.0). It offered a light-weight binding syntax to make an arbitrary HTTP POST to a URL. If the HTTP payload required a specific value to be sent, this allowed the card author to customize what value was sent to the third-party, by using the valueOn and valueOff properties.
It has somewhat less value now given Action.Submit, but it still allows a card author to customize what value gets sent back to them based on the checkbox being selected or not.

Maybe we should get rid of the value property then and replace that with isChecked? Cause it makes things tricky such that when setting value, you have to make sure it corresponds to your valueOn value you specified...
For example if I initially had a toggle that I wanted default checked, I would do the following...
        {
            "type":"Input.Toggle",
            "id": "acceptTerms",
            "title": "I accept the terms and agreements",
            "value": "true"
        }

And then later on if I decided to utilize the valueOn and valueOff properties, I would add those...
        {
            "type":"Input.Toggle",
            "id": "acceptTerms",
            "title": "I accept the terms and agreements",
            "value": "true",
            "valueOn": "accepted",
            "valueOff": "rejected"
        }

But now my checkbox is no longer checked by default! I have to remember to update the value property since I changed the valueOn property... it's an odd and unnecessary dependency.
Additionally, the way it is now makes the booleans inconsistent with the rest of the booleans, since if you're just using the default true/false, you have to use quotes around true, since it's of type string.
        {
            "type":"Input.Toggle",
            "id": "acceptTerms",
            "title": "I accept the terms and agreements",
            "value": "true"
        }

Whereas textblocks use booleans and therefore don't have those quotes around true...
        {
            "type":"TextBlock",
            "text": "This is some text",
            "isSubtle": true
        }

Correct - see issue #784
Heads up - I've built a designer for Adaptive Cards which I will release shortly. This is where in the traditional sense - what I call a paradox bool - can create a logic bug. It isn't so much apparent when you are creating a card declaratively, but when you build out this property dynamically, it can be a mess to trace as to why a value of (string) bool set to True is yielding a False value.

That's awesome! I'll work on getting the schema updated. I'd also love to hear more about the designer, feel free to reach out to<EMAIL_ADDRESS>if you want to share details 😃

Closing as unlikely we'll make any changes to the schema at this point


[terra-folder-tree] Update docs

Summary

This PR improves documentation and examples for terra-folder-tree so that it is easy for consumers to understand how to consume the component.
Testing

This change was tested using:

[ ] WDIO
[ ] Jest
[ ] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[x] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[x] UX review
[ ] Accessibility review
[ ] Functional review

Additional Details

This PR resolves:
UXPLATFORM-9368

Thank you for contributing to Terra.
@cerner/terra

@sycombs  I left a few smaller comments above, please let me know your thoughts on those. In general though, I thought page is well documented, especially the examples. I like how you further explained the individual props needed for each.
I did a quick run through of the examples via keyboard navigation and was able to navigated, expand/collapse, select items as expected.    I also tested them with JAWS on and it all worked as expected and similar to the ARIA site that you linked so good job there. My only question and maybe @eawww  has already addressed this is with the expand/collapse all buttons, once they are selected via keyboard JAWS just says "Enter" there is no confirmation the items were expanded or collapsed.  This shouldn't affect this current PR and am okay with it as is,  I think this is something we can further discuss when Michael does his product validation to determine if we need a follow up story for it.  I just wanted to call it out in the event it had already been discussed somewhere.

@chrismichalewicz I think that's something I called out on a previous PR or something but probably never logged a jira for it. There should be a state-driven aria-live announcement when all collapsible items are either expanded or collapsed.

UXPLATFORM-10089 was logged to address that!

@sycombs, I've pushed my edits to the files, including some edits to the prop tables for consistency. I removed the Features link in the Overview because linking to the subsequent section isn't really necessary. Let me know if any other changes need to be made!


PEBKAC: Not sure how to use resampler on raw data.

Hello. I think I'm not quite sure how to use your library.
I have the following code
stop_recording(&input_stream);
let capacity = (sample_rate * channels * 30) as usize;
let audio_buffer = {
    let mut audio_buffer = Vec::with_capacity(capacity);

    while let Some(mut vec_frames) = thirty_second_audio_buffer.pop() {
        audio_buffer.append(&mut vec_frames)
    }

    let mut pad_the_buffer_with_zeroes = vec![0i16; capacity - audio_buffer.len()];
    audio_buffer.append(&mut pad_the_buffer_with_zeroes);
    log::info!("{:?}", audio_buffer.len());
    audio_buffer
};

let resampler = AudioResampler::builder()
    .source_channel_layout(
        ChannelLayout::from_channels(channels as u32)
            .unwrap()
            .clone(),
    )
    .source_sample_format(SampleFormat::from_str("s16").unwrap())
    .source_sample_rate(48000)
    .target_channel_layout(ChannelLayout::from_channels(1).unwrap())
    .target_sample_format(SampleFormat::from_str("s16").unwrap())
    .target_sample_rate(16000)
    .build()
    .unwrap();

I want to take the audio_buffer which could be in 48khz 2ch, but always pcm_s16le and leverage ffmpeg to downsample to 16khz 1ch, also pcm_s16le. I'm really not sure though how to do this. It looks like it takes audio frames, which are pointers to one sample per channel and a timestamp. But that constructor is private.
The android device generates raw frames, and the input to the next part of the process also takes raw frames, only the sample rate is different. This is why I don't want to write to a file as an intermediate step

Resolved; I may add a minimal example to the examples/ dir, but for now:
To create AudioFrames:
let mut empty_ffmpeg_frame = AudioFrameMut::silence(
    &ChannelLayout::from_channels(channels as u32).unwrap(),
    SampleFormat::from_str(RECORDING_FORMAT_S16_FFMPEG).unwrap(),
    sample_rate as u32,
    sample_rate as usize * seconds,
);
let mut planes = empty_ffmpeg_frame.planes_mut();
let plane_data = planes[0].data_mut();
let mut start = 0;
while let Some(i16le_frames) = thirty_second_audio_buffer.pop() {
    let (_pre, bytes, _post) = unsafe { i16le_frames.align_to::<u8>() };

    let target = &mut plane_data[start..(start + bytes.len())];
    target.copy_from_slice(bytes);

    start = bytes.len();
}
empty_ffmpeg_frame.freeze()

Then just pass the whole thing to the resampler
let resampler = AudioResampler::builder()
        .source_channel_layout(ChannelLayout::from_channels(channels as u32).unwrap())
        .source_sample_format(SampleFormat::from_str(RECORDING_FORMAT_S16_FFMPEG).unwrap())
        .source_sample_rate(sample_rate as u32)
        .target_channel_layout(ChannelLayout::from_channels(RESAMPLE_TARGET_CHANNELS).unwrap())
        .target_sample_format(SampleFormat::from_str(RECORDING_FORMAT_S16_FFMPEG).unwrap())
        .target_sample_rate(RESAMPLE_TARGET_HZ)
        .build()
        .unwrap()
resampler.push(ffmpeg_audio_frame).unwrap();
let frame = resampler.take().unwrap().unwrap();
let raw_data = frame.planes()[0].data();


Add optional support for Serde

Fixes #5
Edit: Surprisingly, Deserialize/Serialize aren't working for structs that are relying on derives. This matter needs more investigation.

I'll merge this as-is for the sake of simplicity and so you can get the credit for it, but I'm gonna make a few changes to the file structure for it / the TOML settings / e.t.c right afterwards.
Thanks a lot!

Thanks!


Parsers for HTTP Query and Headers should provide more error details on invalid inputs

Currently, for invalid values in HTTP headers and queries, it throws an exception without any details.
It should provide some related info such as description and parameter name.
This issue is created from this comment.

This is a great change, been meaning to ask for it for a while.
Can the createInvalidQueryErrors() and createParamErrors() methods be protected instead of private? This would allow the class to be extended and just those methods overriden to allow custom errors to be returned?

@lindyhopchris done

released in 1.0.5


Update references to LGTM

🐛 Summary
LGTM was a code-scanning tool that no-longer exists under that name. It is now built into GitHub Actions under the name, CodeQL. References to LGTM are in two files: /README.md and /languages/python/README.md.
Proposed Solution

Implement CodeQL scanning in all cisagov repositories (if not already done).
Update README files to replace references to LGTM with CodeQL.

Source
The next step for LGTM.com: GitHub code scanning!
About code scanning with CodeQL

I believe this is already taken care of in #79, which is awaiting re-review by @mcdonnnj.

Great! Thanks! I'm closing this issue since it is already covered in #79.


API Change

Hello,
We’re pushing a few changes to ensure continued service, higher availability, and lower latencies for our APIs.
What’s changing?
On 1st June 2022 at 00:00 UTC, we’re going to deprecate some endpoints used to reach our Geocoding, Routing & Timezones APIs. These include older & root domain endpoints such as us1.locationiq.org/v1, eu1.locationiq.org/v1, locationiq.org/v1, and locationiq.com/v1.
Starting 1st June, our Geocoding, Routing & Timezone APIs will only respond to requests sent to these two endpoints - https://us1.locationiq.com/v1, and https://eu1.locationiq.com/v1.
There are no changes to other APIs:
Autocomplete API requests - api.locationiq.com/v1
Static Maps - maps.locationiq.com/v3
Map Tiles - tiles.locationiq.com/v3
What you need to do
If you’re using one of our new endpoints - you’re good! If you’re using any of the older endpoints, please update your request URLs to the above-mentioned endpoints.
Please check out our documentation page for more information about our APIs. Just hit reply if you have any questions, we’d be happy to help.

Based on the options selectable from GEOCOORDS_PROVIDERS, this should be fine.
https://github.com/sopel-irc/sopel-weather/blob/b17bacf982625daa81e99af3d675d593f16e45f5/sopel_modules/weather/weather.py#L28-L33


Style auto import dosen't work

Create an app with CRA:
npx create-react-app demo
cd demo
yarn eject

Install arco design and the plugin:
yarn add @arco-design/web-react
yarn add -D @arco-design/webpack-plugin

Modify  config/webpack.config.js to add the plugin:
...
const ArcoWebpackPlugin = require('@arco-design/webpack-plugin');
...
module.exports = function (webpackEnv) {
  ...
  return {
    ...
    plugins: [
      ...
      new ArcoWebpackPlugin(),
    ].filter(Boolean),
    ...
  };
};

Change src/App.js content to:
import { Button } from '@arco-design/web-react';

function App() {
  return (
    <Button>Button</Button>
  );
}

export default App;

Run with yarn start, it seems that style auto import dosen't work:

@wildseeder Hello, in order to help you solve the problem faster, please provide a minimal project to reproduce the problem。

@wildseeder Hello, in order to help you solve the problem faster, please provide a minimal project to reproduce the problem。

@kirazxyun the repo following steps above: https://github.com/wildseeder/arco-webpack-plugin-style-issue

@wildseeder Hello, in order to help you solve the problem faster, please provide a minimal project to reproduce the problem。

@kirazxyun the repo following steps above: https://github.com/wildseeder/arco-webpack-plugin-style-issue

Sorry to reply you so late.  You need support the CSS preprocessor  less in your project.

Sorry to reply you so late. You need support the CSS preprocessor less in your project.

@kirazxyun Thanks for your answer, it would be nice to have the corresponding description in the documentation.


fix version (rust-toolchain)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 42.832%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
729

Relevant Lines:
1702

💛  - Coveralls


-o "..\  Becomes ERROR: [WinError 2] The system cannot find the file specified: '..\ (Automatic Prepended \ in filename output reguardless if specified)

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
Possibly a bug in handling/recognizing  ..\  in output filenames. It appears to automatically prepend a \ on to all filenames regardless if the output filename specified includes  \
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
C:\youtube-DL\APP>yt-dlp.exe -vU --restrict-filenames -o "..\%(title).100s (%(id)s).%(ext)s" --merge-output-format mp4 --postprocessor-args "-strict experimental" -f "bestvideo+bestaudio/best" --embed-thumbnail --add-metadata --cookies-from-browser firefox --cookies tiktok_cookies.txt "https://www.tiktok.com/t/ZTNNef8GN/"
[debug] Command-line config: ['-vU', '--restrict-filenames', '-o', '..\\%(title).100s (%(id)s).%(ext)s', '--merge-output-format', 'mp4', '--postprocessor-args', '-strict experimental', '-f', 'bestvideo+bestaudio/best', '--embed-thumbnail', '--add-metadata', '--cookies-from-browser', 'firefox', '--cookies', 'tiktok_cookies.txt', 'https://www.tiktok.com/t/ZTNNef8GN/']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds [a8520244b] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg git-2020-08-28-ccc7120, ffprobe git-2020-08-28-ccc7120
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.06.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
Extracting cookies from firefox
[debug] Extracting cookies from: "C:\Users\Brian\AppData\Roaming\Mozilla\Firefox\Profiles\ebmleyw3.default\cookies.sqlite"
Extracted 2606 cookies from firefox
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
WARNING: Post-Processor arguments given without specifying name. The arguments will be given to all post-processors
[debug] Loaded 1823 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds)
[vm.tiktok] Extracting URL: https://www.tiktok.com/t/ZTNNef8GN/
[vm.tiktok] ZTNNef8GN: Downloading webpage
[TikTok] Extracting URL: https://www.tiktok.com/@ryanjosephnetwork1/video/7378788729533549866?_t=8nQzVaRgfMM&_r=1
[TikTok]<PHONE_NUMBER>533549866: Downloading webpage
[debug] [TikTok] Found universal data for rehydration
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info]<PHONE_NUMBER>533549866: Downloading 1 format(s): bytevc1_1080p_352374-1
Deleting existing file ..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).jpg
[info] Downloading video thumbnail 2 ...
[info] Writing video thumbnail 2 to: ..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).jpg
[debug] Invoking http downloader on "https://v19-webapp-prime.us.tiktok.com/video/tos/useast5/tos-useast5-pve-0068-tx/o0BGy8QSIiZUixMB1uzCmB8AME9bprNfwAnihI/?a=1988&bti=ODszNWYuMDE6&ch=0&cr=3&dr=0&lr=unwatermarked&cd=0%7C0%7C0%7C&cv=1&br=688&bt=344&cs=2&ds=4&ft=4KJMyMzm8Zmo0GnD5-4jVBOZdpWrKsd.&mime_type=video_mp4&qs=15&rc=aTY2NDU5NGkzaDo6OmllNEBpM3NscHY5cjZlczMzZzczNEA1M2JjNi0zXmMxYTRfYV8vYSM2am5nMmRjZmdgLS1kMS9zcw%3D%3D&btag=e00098000&expire=1719173845&l=202406231412356A31BFEAC2BC1D36E843&ply_type=2&policy=2&signature=e0a873b476e6b54e9a3dbe74d50be70a&tk=tt_chain_token"
[download] ..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4 has already been downloaded
[download] 100% of   13.54MiB
[Metadata] Adding metadata to "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i "file:..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4" -map 0 -dn -ignore_unknown -c copy -write_id3v1 1 -metadata "title=original sound" -metadata "date=20240610" -metadata "description=#usa #cnn #biden #trump #trend #foxnews #trump #trend " -metadata "synopsis=#usa #cnn #biden #trump #trend #foxnews #trump #trend " -metadata "purl=https://www.tiktok.com/@ryanjosephnetwork1/video/7378788729533549866?_t=8nQzVaRgfMM&_r=1" -metadata "comment=https://www.tiktok.com/@ryanjosephnetwork1/video/7378788729533549866?_t=8nQzVaRgfMM&_r=1" -metadata "artist=Ryan Joseph" -movflags +faststart -strict experimental "file:..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).temp.mp4"
[EmbedThumbnail] mutagen: Adding thumbnail to "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4"
WARNING: unable to embed using mutagen; None
[EmbedThumbnail] mutagen was not found. Falling back to AtomicParsley
[EmbedThumbnail] atomicparsley: Adding thumbnail to "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4"
[debug] AtomicParsley command line: AtomicParsley "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4" --artwork "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).jpg" -o "..\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).temp.mp4" -strict experimental
WARNING: Unable to embed thumbnails using AtomicParsley;
ERROR: [WinError 2] The system cannot find the file specified: '..\\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).temp.mp4' -> '..\\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4'
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1605, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1761, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1820, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2993, in process_video_result
  File "yt_dlp\YoutubeDL.py", line 3537, in process_info
  File "yt_dlp\YoutubeDL.py", line 3719, in post_process
  File "yt_dlp\YoutubeDL.py", line 3701, in run_all_pps
  File "yt_dlp\YoutubeDL.py", line 3679, in run_pp
  File "yt_dlp\postprocessor\common.py", line 23, in run
  File "yt_dlp\postprocessor\common.py", line 128, in wrapper
  File "yt_dlp\postprocessor\embedthumbnail.py", line 218, in run
FileNotFoundError: [WinError 2] The system cannot find the file specified: '..\\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).temp.mp4' -> '..\\usa_cnn_biden_trump_trend_foxnews_trump_trend<PHONE_NUMBER>533549866).mp4'

The \\ is just the python string representation and expected. Since python allows for escapes like \n, an actual \ gets represented as \\, which explains why it is always added. It has nothing to do with the error you are experiencing. See this:
>>> print('..\\%(title).100s (%(id)s).%(ext)s')
..\%(title).100s (%(id)s).%(ext)s

The failed embedding was a chain of errors that resulted in all embedding attempts failing:

First mutagen embedding is tried. To do this, compat.imghdr is invoked to determine the file type. Since that module is not up to date it incorrectly judges the jpeg thumbnail, returning None and causing a KeyError(None). mutagen is skipped and the cryptic error WARNING: unable to embed using mutagen; None output.
AtomicParsley is tried next, however since you passed --postprocessor-args "-strict experimental" that command fails as the -strict experimental gets passed to all postprocessors, and AtomicParsley does not understand that flag

Side note: You should use --postprocessor-args "ffmpeg:-strict experimental" to only pass it to ffmpeg or, preferably, upgrade your ffmpeg so the flag is not needed at all

Now another error comes into play, where after AtomicParsley fails with an errorlevel, success is not set to False, so embedding with ffmpeg is not tried next
The non-existent temporary file will now be moved to overwrite the regular file

The last part ultimately resulted in the error you saw:
>>> os.replace(R"..\file.temp.mp4", R"..\file.mp4")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [WinError 2] The system cannot find the file specified: '..\\file.temp.mp4' -> '..\\file.mp4'

A PR is about to be opened for all these to be fixed. Make sure to update ffmpeg and no longer pass --postprocessor-args


add option in model generation to output all definitions regardless of usage

Attempt at solving #1679
I have manually tested this will output all the expected go code for definitions defined in a swagger spec where paths: {}.
This is my first contribution, so apologies if I've missed anything. I'm guessing I might need to add some tests, any pointers into that would be appreciated.
Signed-off-by: dan-j<EMAIL_ADDRESS>

FYI in case anybody noticed my previous MR.. It was on dan-j:master, and the CI checks failed but appeared to not be related to my changes. I noticed other PRs which pass were created from a branch different from master so I've recreated it on a different branch


⚠ Bofrost.at has degraded performance

In 0ab34f9, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1576 ms

Resolved: Bofrost.at performance has improved in 09a1afd.


Not possible to run Rspec on only branch with some basic specs.

Looks like it's not possibile to run specs on moduler branch (the only one with some features specs added). It seems that it doesn't find the login_success method. I think it'd be useful to make rspec work and to add some guidelines especially on how to integrate rspec for the FE of a camaleon install so that people using camaleon could have better understading on what is going on under the hood and have at least some basic tests (unit & integration) that they can use with some real rails application.

For now please ignore the test folder, because I am trying to integrate my small tests in the gem but I don't know where to put the rspec configurations.

Thanks for your quick reply, so basically are you going to drop rspec support in favour of test unit or some other suite? I think that the issue with the missing login_success method can be easily fixed (guess we need to wrap common.rb into a module and include it when rspec runs - like you did in wait_for_ajax
Fact is that even after fixing this type of issue it's not clear how and what to require, since there are several custom initializers in the config folder of the gem itself and feature specs always complain about missing something either related to rails (e.g activerecord) or camaleon itself.
Also, at this point when do you think are you going to release those tests? At least a basic (working) setup of a test suite on camaleon itself would be enough to let other devs (like me) help you debug and add other features :smile:
Also, are you going to provide some sort of guidelines (or tests) on how to integrate a test suite on a rails app that is using camaleon? Becasue I think that would also be useful.
I think that at this point it would be better to completely remove rspec from the moduler branch because that could be misleading, or at least it was for me: I've deliberately started working with V2 because I thought I could have some test support :sob:
Keep me posted ... :smile:

I think a lot of refactoring needs to happen in order to make a test suite happen.


Catch websocket disconnect event

Is your feature request related to a problem? Please describe.
I am using WebSocketHubProvider.
There is a few cases i cant catch/handle:

client close socket and i cant find method wich nameko calls.
client quit without closing a socket and nameko throws exception.

Describe the solution you'd like
Provide a  way to catch disconnect events.

Sorry for the slow reply here. I'm not very familiar with the websocket implementation, but I think at least the "client quit" case has come up before. Have you seen this post on the mailing list and the linked issue?
I would welcome a PR with either better docs, an improved implementation or "websocket rpc" implementation as discussed in those posts.


refactor makefile

adds QOL improvements to makefile targets to ease development and
debugging
DM-#10

Closes #10


GWR Model can not run on Polygon feature with error Execute failed: x attribute access only provided for Point geometries

Execute failed: x attribute access only provided for Point geometries
try this data: https://geodacenter.github.io/data-and-lab/data/airbnb.zip
with GeoFile reader, and GWR model

already solved


Adding amp-agent targets to the Makefile

This PR adds amp-agent targets in the Makefile by mimicking what already exists for other binaries, namely:

build-agent: Build the alpine binary and then build the docker image
rebuild-agent: Does the same, expect it cleans the binary first
clean-agent: Cleans the binary

Note the amp-agent code/binary hasn't been updated for now

@bquenin I thought about that as well when reviewing the PR earlier, but decided not to mention it to @freignat91 for now. I figure we'll do this anyway to migrate amp -> atomiq and it might as well be part of one coordinated update. I actually was going to ask you if you wanted to tackle this since I noticed your attention to detail is quite good when renaming/refactoring things. Let's merge this PR for him, but let's talk about renaming tomorrow.


Something is Messed Up -- Multiple Errors & Issues

What version of Music Assistant has the issue?
2022.8.1
The problem
I'm not sure when these issues started, either with the 2022.8.1 Core update or 2022.8.1 MA Update but I'm now having functional and UI issues, lots of them.  So many in fact, I'm not sure where to start.

Intermittent Errors and failures when trying to start playing at track.
1a) Sometime if I move to Played Items I can get a track to play, and that sometimes allows the Queue to start working
Play Head (Transport Section) is often out of sync with what shows playing.
The Play Now does not work if another track is playing.
If I Select Play Next in a Queue, after the current track completes, the entire MASS system stops working.
When starting a play I'm getting a failed login attempt. I don't understand this, but it ALWAYS coincides with doing something in MASS (Login attempt or request with invalid authentication from homeassistant.localdomain (<IP_ADDRESS>). See the log for details.)

There are other issues, but like I said, I'm not sure what the heck is going on.  I hate to dump my MASS setup and start over again, but these don't seem like 'standard' problems.

How to reproduce
Using the system.
Relevant log output
This error originated from a custom integration.
Logger: aiohttp.server
Source: custom_components/mass/panel.py:92
Integration: Music Assistant (documentation, issues)
First occurred: 8:53:30 AM (2 occurrences)
Last logged: 8:54:08 AM
Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
resp = await request_handler(request)
File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
resp = await handler(request)
File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 82, in ban_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
result = await result
File "/config/custom_components/mass/panel.py", line 92, in get
data = await mass.metadata.get_thumbnail(url, size=size)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/metadata/metadata.py", line 260, in get_thumbnail
thumbnail = await create_thumbnail(self.mass, path, size)
File "/usr/local/lib/python3.10/site-packages/music_assistant/helpers/images.py", line 45, in create_thumbnail
img_data = await get_image_data(mass, path)
File "/usr/local/lib/python3.10/site-packages/music_assistant/helpers/images.py", line 38, in get_image_data
raise FileNotFoundError(f"Image not found: {path}")
FileNotFoundError: Image not found: http://<IP_ADDRESS>:8123/media/media/music/Tobymac/Hits Deep Live (2016) [MP3]/12 Undeniable (feat. Colton Dixon) [Live].mp3
AND
Logger: homeassistant
Source: /usr/src/homeassistant/homeassistant/runner.py:96
First occurred: 8:56:54 AM (2 occurrences)
Last logged: 8:57:05 AM
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 411, in next
await self.play_index(next_index)
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 535, in play_index
stream = await self.queue_stream_start(
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 712, in queue_stream_start
stream = await self.mass.streams.start_queue_stream(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/streams.py", line 308, in start_queue_stream
streamdetails = await get_stream_details(self.mass, first_item, queue.queue_id)
File "/usr/local/lib/python3.10/site-packages/music_assistant/helpers/audio.py", line 241, in get_stream_details
full_item = await mass.music.get_item_by_uri(queue_item.uri)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/music.py", line 231, in get_item_by_uri
return await self.get_item(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/music.py", line 254, in get_item
return await ctrl.get(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/tracks.py", line 45, in get
track = await super().get(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 148, in get
db_item = await self.get_db_item_by_prov_id(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 348, in get_db_item_by_prov_id
return await self.get_db_item(provider_item_id)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 335, in get_db_item
raise MediaNotFoundError(f"Album not found in database: {item_id}")
music_assistant.models.errors.MediaNotFoundError: Album not found in database: 14479
AND
Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/mass/websockets.py:1188
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 9:10:39 AM (2 occurrences)
Last logged: 9:10:50 AM
[139695236201296] Error handling message: Unknown error (unknown_error)
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 27, in _handle_async_response
await func(hass, connection, msg)
File "/config/custom_components/mass/websockets.py", line 114, in async_get_mass_func
await orig_func(hass, connection, msg, mass)
File "/config/custom_components/mass/websockets.py", line 1188, in websocket_play_media
await player_queue.play_media(
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 275, in play_media
await self.play_index(cur_index)
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 535, in play_index
stream = await self.queue_stream_start(
File "/usr/local/lib/python3.10/site-packages/music_assistant/models/player_queue.py", line 712, in queue_stream_start
stream = await self.mass.streams.start_queue_stream(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/streams.py", line 308, in start_queue_stream
streamdetails = await get_stream_details(self.mass, first_item, queue.queue_id)
File "/usr/local/lib/python3.10/site-packages/music_assistant/helpers/audio.py", line 241, in get_stream_details
full_item = await mass.music.get_item_by_uri(queue_item.uri)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/music.py", line 231, in get_item_by_uri
return await self.get_item(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/music.py", line 254, in get_item
return await ctrl.get(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/tracks.py", line 45, in get
track = await super().get(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 148, in get
db_item = await self.get_db_item_by_prov_id(
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 348, in get_db_item_by_prov_id
return await self.get_db_item(provider_item_id)
File "/usr/local/lib/python3.10/site-packages/music_assistant/controllers/media/base.py", line 335, in get_db_item
raise MediaNotFoundError(f"Album not found in database: {item_id}")
music_assistant.models.errors.MediaNotFoundError: Album not found in database: 1
Additional information
No response
What version of Home Assistant Core are your running
2022.8.1
What type of installation are you running?
Home Assistant OS
On what type of hardware are you running?
Windows

Did you do something to your database file ?
Your log states MediaNotFoundError's on database id's so those id's are no longer there.

My only action was updating both HA and MASS.

OK, this looks like a browser cache issue where you're still using the previous version of the frontend.
Can you try to force-refresh the browser cache ?

Thanks for the help and the clue...
Clearing the cache didn't have any effect. However, at this point I decided to

Shutdown MASS
Delete the music_assistant.db
Restart MASS
Force a Full System.. (probably didn't have to do this part).

This fixed the problems. My guess is that something damaged the db during the HA or MASS update.

Thanks Marcel has been working hard to eliminate the need to rebuild the db but yes during this rapid development phase sometimes it might be the only way.


Cross-site Scripting (XSS) - Reflected

Hey,
We've found a Cross-site Scripting (XSS) - Reflected at https://httpbingo.org
POC
navigate to URL :
https://httpbin.org/base64/ZXhhbXBsZS5vcmciPjxzdmcvb25sb2FkPXByb21wdCgneHNzJyk+
xss will pop up
Impact
attacker is able to execute javascript code on users

the go port solved this by ensuring a text/plain is always returned rather then a text/html
https://github.com/mccutchen/go-httpbin/pull/68