ClientAuthenticationError: (PermissionDenied) Principal does not have access to API/Operation.

Please provide us with the following information:

This issue is for a: (mark with an x)
- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

My organization already has a resource group with the necessary resources in them, i.e. app service, ai search, document intelligence etc.
I'm owner of the resource group but I do not have the permission to create a new resource group - hence I cannot use azd up. My plan is to create env variables locally, then index the documents and finally az webapp deploy the code to my existing app service.
So, locally, first I ran azd auth login and then azd env set for all the variables listed here and in main.parameters.json
Next I ran .\scripts\prepdocs.ps1 which runs prepdocs.py to create an index in azure ai search. But I get this error - Principal does not have access to API/Operation.
What am I missing here? I presume it has to do something with role assignments as I did not run azd up?

@pamelafox any inputs from you will be highly appreciated!
Any log messages given by the failure
Loading azd .env file from current environment
Creating python virtual environment ".venv"
Installing dependencies from "requirements.txt" into virtual environment
Running "prepdocs.py"
./app/backend/prepdocs.py "\data*" --verbose --subscriptionid subscriptionid --storageaccount storageaccount --container container --storageresourcegroup storageresourcegroup --searchservice searchservice --index index  --openaihost "" --openaimodelname ""  --openaiservice openaiservice --openaideployment openaideployment --openaicustomurl ""  --openaiorg "" --documentintelligenceservice documentintelligenceservice
\app\backend\prepdocs.py:221: SyntaxWarning: invalid escape sequence '*'
epilog="Example: prepdocs.py '.\data*' --storageaccount myaccount --container mycontainer --searchservice mysearch --index myindex -v",
Using local files: \data*
Ensuring search index index exists
Search index index already exists
Ingesting 'Test.pdf'
Extracting text from '\data\Test.pdf' using Azure Document Intelligence
Traceback (most recent call last):
File "\app\backend\prepdocs.py", line 479, in
loop.run_until_complete(main(ingestion_strategy, setup_index=not args.remove and not args.removeall))
File "C:\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "\app\backend\prepdocs.py", line 215, in main
await strategy.run()
File "\app\backend\prepdocslib\filestrategy.py", line 84, in run
sections = await parse_file(file, self.file_processors, self.category, self.image_embeddings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "\app\backend\prepdocslib\filestrategy.py", line 26, in parse_file
pages = [page async for page in processor.parser.parse(content=file.content)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "\app\backend\prepdocslib\pdfparser.py", line 54, in parse
poller = await document_intelligence_client.begin_analyze_document(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv\Lib\site-packages\azure\core\tracing\decorator_async.py", line 94, in wrapper_use_tracer
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv\Lib\site-packages\azure\ai\documentintelligence\aio_operations_operations.py", line 3241, in begin_analyze_document
raw_result = await self._analyze_document_initial(  # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".venv\Lib\site-packages\azure\ai\documentintelligence\aio_operations_operations.py", line 130, in _analyze_document_initial
map_error(status_code=response.status_code, response=response, error_map=error_map)
File ".venv\Lib\site-packages\azure\core\exceptions.py", line 161, in map_error
raise error
azure.core.exceptions.ClientAuthenticationError: (PermissionDenied) Principal does not have access to API/Operation.
Code: PermissionDenied
Message: Principal does not have access to API/Operation.
Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

azd version 1.9.6

Mention any other details that might be useful

Thanks! We'll be in touch soon.

If you haven't run azd up, then you need to also set all the roles that are set in infra/main.bicep. You should be able to set them with this script:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/scripts/roles.ps1

I have the same issue when running this on azure container app an asking a question. The  mentioned spi roles shall not be needed for the server app to execute , are they?


"Back to application" during logout redirects wrong

Description - What happened? *
When I log out as a logged-in user and click on "Back to application" I am redirected to the "landing page" suggesting a successful log-out, although I am still logged in (click on "Log in" to prove).
Expected Behavior *
Instead, I should be redirected to the Dashboard.
Observed Behavior *
see Description
Steps to Reproduce
No response
Context Information
No response
Relevant log output
No response
Screenshots
No response

I did some investigating, so I'll share the context here.
When a user presses the "Log out" button and then does not confirm the logout but rather clicks on "Back to application", the user gets redirected to the Portal's unauthenticated portal page.
At this stage, the user is still logged in according to Keycloak and has a valid access token.
But:
The pings to /user-info return 401 Unauthorized which hints at the backend revoking access on the first "Log out" button and thus not redirecting the user to the authenticated part of the FE.
This is further confirmed by setting the redirect URL to PORTAL_FQDN/mds-home and still seing the user getting stuck on the unauthenticated landing page.


Update Quotes.js

you did not read contribution guideline properly.

Sorry for the inconvenience . I have star the repository now.

seems ok 🆗
merged✅


2021-01-10 Version 3.6.1: Updated tests

Codecov Report

Merging #59 (c71b55a) into master (5088194) will increase coverage by 0.71%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master      #59      +/-   ##
============================================
+ Coverage     70.33%   71.05%   +0.71%
  Complexity        2        2
============================================
  Files            67       67
  Lines          1824     1824
  Branches        139      139
============================================
+ Hits           1283     1296      +13
+ Misses          455      447       -8
+ Partials         86       81       -5

Impacted Files
Coverage Δ
Complexity Δ

...l/jaman/hashchecker/logic/history/HistoryItem.java
100.00% <0.00%> (+4.00%)
0.00% <0.00%> (ø%)

.../hashchecker/logic/history/ui/HistoryFragment.java
90.90% <0.00%> (+6.81%)
0.00% <0.00%> (ø%)

...ogic/history/ui/loader/HistoryItemsLoaderTask.java
100.00% <0.00%> (+7.69%)
0.00% <0.00%> (ø%)

...er/logic/hashcalculator/jdk/JdkHashCalculator.java
92.30% <0.00%> (+30.76%)
0.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5088194...c71b55a. Read the comment docs.

Codecov Report

Merging #59 (c71b55a) into master (5088194) will increase coverage by 0.71%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master      #59      +/-   ##
============================================
+ Coverage     70.33%   71.05%   +0.71%
  Complexity        2        2
============================================
  Files            67       67
  Lines          1824     1824
  Branches        139      139
============================================
+ Hits           1283     1296      +13
+ Misses          455      447       -8
+ Partials         86       81       -5

Impacted Files
Coverage Δ
Complexity Δ

...l/jaman/hashchecker/logic/history/HistoryItem.java
100.00% <0.00%> (+4.00%)
0.00% <0.00%> (ø%)

.../hashchecker/logic/history/ui/HistoryFragment.java
90.90% <0.00%> (+6.81%)
0.00% <0.00%> (ø%)

...ogic/history/ui/loader/HistoryItemsLoaderTask.java
100.00% <0.00%> (+7.69%)
0.00% <0.00%> (ø%)

...er/logic/hashcalculator/jdk/JdkHashCalculator.java
92.30% <0.00%> (+30.76%)
0.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5088194...c71b55a. Read the comment docs.


Unique count on duplicated data set.

I got a dataset with a lot of duplicated rows. It comes from a master table with a few one to many related tables. When we links these tables together to product a simple table/spreadsheet. There are a lof duplications.
In Excel pivot table, we can use the unique count to remove duplications. The same feature is avaiable in PowerBI.
Do you have the same unique count in Pygwalker?

Hi, @dickhfchan  sorry for the late reply.
It's not sure to me whether computed fields solve your needs.
example:

Hi @longxiaofei , it works.


kata-env shows incorrect debug values for runtime+hypervisor

The output of kata-env incorrectly always shows Debug = false for both the runtime and the hypervisor.
The other elements all show correct debug values and although the runtime+hypervisor debug value is incorrect, all the debug options in configuration.toml are honoured.

@sboeuf, @egernst - I think this should go into stable so label added.

@jodh-intel makes sense!


Include 'self' in the list of returned links

For testing a web application, I need to be able to verify the existence of the 'self' link.
The uri of the self link is stored as the uri of the HALNavigator object. But for testing link availability it should also be included in the list of links. Is there any reason not to include it?
I note that in '_update_self_link' you never update the uri of the HALNavigator object (guess that would break the APICore caching...?) but that does mean that if the 'self' link changes during the lifetime of a resource, then the HALNavigator uri would be out of synch. On the other hand, I'm not sure what would be the correct thing to do in this case.

Coverage remained the same at 95.34% when pulling d6fa29c751a8313a02911f068a5cc0f35fe5d9fd on bbsgfalconer:show_self_link into 4358a8f8de36a92ef5a8c6104ba8d6d8eeca7728 on deontologician:next.

The uri is always the same for a given resource since it's the identity of the resource. It wouldn't make sense to be able to change it, instead you get a link to a new resource.
I think this looks fine. Again I'd want a new test explicitly testing the difference between the new logic and old

Ah, I just also realized why this probably isn't a good idea. Basically, since we convert all links into navigators, it means every navigator would hold a link to itself. We could do it, because I believe python's reference counting has the ability to check for cycles like this, but I'm not sure it's a great idea

On the other hand, (at least for the API I am working with) I find that most Navigators already hold links to themselves...
For example, if a hal representation includes links used to PUT, PATCH or DELETE to the resource itself, those result in a Navigator with links referencing itself.
The API I am working with uses named semantic links for PUT and DELETE... I have not experienced any problems... but perhaps I'm not looking closely enough.

That's a fair point. We don't do anything elsewhere to prevent a cycle like this


Worksheet length ?

Is there any way to know from a worksheet, the last row that has a value ?
I want to add new values to new rows but first I will need to know witch row is the last one

Try something like:
const lastUsedRowNumber = sheet.usedRange().endCell().rowNumber();


eq-180 RoutingStatement component

What is the context of this PR?
Trello card

moved existing routing-specific files into 'routing' directory.
added RoutingStatement component that encapsulates 'AND' button and layout.

How to review
Check storybook against AC

 Routing Statement Component (3)


backlink on an item doesn't render if there's no description

See #3 for more background, especially https://github.com/jtracey/journal-backlinks/issues/3#issuecomment-1120548476 for context on why this is happening.

I believe this was fixed as part of 17f6612c504e1892dd7d66a3ae1f390aa524ec5c fixing #10. We can reopen if I missed something.


support implementation_deps on cuda_object and cuda_library

implementation_deps are an option with cc_library and allow you to add deps that don't propagage
https://bazel.build/reference/be/c-cpp#cc_library.implementation_deps

In rdc case, objects from deps should always propagate for dlink, and in non-rdc case, objects does not propagate

I didn't put together an issue yet. But I was also noticing that library dependencies set on cuda_objects don't get linked in.
For our unit testing framework catch2, I had to add the dependencies again to our cc_test target even though they're already specified on the cuda_object in order to get linking for the tests to work:
https://github.com/spaceandtimelabs/blitzar/blob/main/bazel/sxt_build_system.bzl#L61-L62

That is because cuda_objects only provides a CcInfo with only compilation_context, but no linking context. As the original design, cuda_objects is just compilation actions, whose actifacts are to be fed into a cuda_library for final dlink. Move it to implementation_deps also clarify the purpose here.

That is because cuda_objects only provides a CcInfo with only compilation_context, but no linking context. As the original design, cuda_objects is just compilation actions, whose actifacts are to be fed into a cuda_library for final dlink.

I don't think that scales well to larger projects with external dependencies. In our case, nearly everything in our project is a cuda_object and we only combine it into a cuda_library at the end when we're linking the shared library:
https://github.com/spaceandtimelabs/blitzar/blob/main/cbindings/BUILD#L10-L21
It works ok right now, as we don't have many external dependencies. But if we did start using external library, we'd have to keep track of everything used and add them as dependencies to the final cuda_library -- that could start to become a maintenance burden.
Is there any reason why the cuda_objects can't carry the linking dependencies?

Is there any reason why the cuda_objects can't carry linking dependencies?

It is not "can't" per se, it is just not implemented.

I don't think that scales well to larger projects with external dependencies. In our case, nearly everything in our project is a cuda_object...

I am not worried about this. The reason for cuda_objects all over around in your repo is because cuda_library does not work for shared library linking. once we fix them, you should be able to dlink multiple times. But we can impl it anyway.

Another problem arised when it comes to transitive rdc (allow cuda_library to consume cuda_library and propagate the rdc objects). Previously, it was pretty simple and rdc=True was designed to mark a dlink step in the rule. Then the produced archive is a self contained and is turned back to normal library. Now we need handle the transitive device link issue.


Doubt in the code of autoencoder.py file

Hi Morvan,
Very humbly requesting you to correct my doubt.
In the decoding section of your code, you didn't take the transpose of previous weight matrices.
Why you did so?

This file: tutorials/tensorflowTUT/tf21_autoencoder/full_code.py

I took transpose of the first two weight matrices to be used in the decoding section.But not getting good results when compared to the result produced by your code( code without transpose of weight matrices)


fix: zh docs mislead to en link.

original link /docs/getting-started-guides/minikube is redirecting to /docs/setup/, which I think is not proper. so I change it to /zh/docs/setup/learning-environment/minikube/.

/lgtm
/approve
It would be great if you can fix the English link as well.

@tengqm just checked that en link is correct as /docs/setup/learning-environment/minikube/.


Some TLC

Cleaning up old, large binary blobs from the git history with bfg resulted in some odd file artifacts. This PR removed those and also adds a fresh lockfile with up to date Makefile commands.

In my (failed) attempt to migrate to Parcel 2.0, I left type=module declarations in the script tags used to bootstrap WebDash in the HTML file. This seems to have confused Parcel 1.x by raising an exception Uncaught ReferenceError: parcelRequire is not defined visible in the console. Not affecting the usability of WebDash, but an issue nevertheless. I removed those declarations from the HTML file.


sbt should try to recover from a corrupted target/ directory

Out of ~210 students starting a class at EPFL today using sbt 1.4.7, at least two had issues related to corrupted files in their target/ directory which prevented them from using sbt until I told them to clean that directory:

[error] java.lang.NoClassDefFoundError: $643da8ee4d202b04c2da$
[error]     at $7024695f5b734dee7409$.$anonfun$$sbtdef$1(/Users/oskar/EPFL/CS-206/cs206-example/build.sbt:12)
[error]     at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:46)
[error]     at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error]     at sbt.Execute.work(Execute.scala:291)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]     at java.lang.Thread.run(Thread.java:748)

I have no idea how this happens (hardware issue?) but since it doesn't seem to be that uncommon, it would be great if sbt had a fallback mode where it would delete its own target and restart instead of exiting with an error because it failed to start.

@smarter Thanks for the report. I guess first thing we should look into is make sure the metabuild automatically gets invalidated when sbt version changes.

I've also seen had two students now where sbt failed to find its own classes:

I was able to confirm at least in one case that deleting ~/.sbt fixed the problem.


[macos] watchPath seems to be false

Describe the bug
I serve lot of files (about 430) and certain files are not watched for some obscur reason.
I added some log before fsevent.watch, for example for these two files (they are symlink):

<EMAIL_ADDRESS><EMAIL_ADDRESS>
The watch folder is /Users/julienrousseau/Documents/insight/monorepo/node_modules/@doubletrade after:
  if (couldConsolidate(parentPath)) {
    watchPath = parentPath;
  }

But only change on dt-notification.ts emit change event.
I don't understand why...
If I comment this:
  if (couldConsolidate(parentPath)) {
    watchPath = parentPath;
  }

All my files are correctly watched.
Versions (please complete the following information):

Chokidar version ^3.4.0
Node version 12/13/14
OS version: Last macos

To Reproduce:
I don't know ...

More precision.
Two same folder but one is symlink:
/Users/julienrousseau/Documents/insight/monorepo-new/packages -> Launch event on /Users/julienrousseau/Documents/insight/monorepo-new/packages/xx/x.js
/Users/julienrousseau/Documents/insight/monorepo-new/node_modules/@doubletrade -> Doesn't launch event on<EMAIL_ADDRESS>

Can ben solved by using the real path instead of symlink path, is it a good choice?
https://github.com/paulmillr/chokidar/pull/1041/files

the new tests at my mac keep failing. @RoXuS can you check this?

hi @paulmillr  on my mac it is ok?
Do you do something that special??

@RoXuS can you test it without .only?

yeah I get this error

Tests doesn't seem to be unitary?
I check how to do that.
I have some other errors, you too?

yep...

these errors were present before my add?

ok on the previous commit I have these failed too...
I'm trying to fix my test when we execute all tests.

I open a new PR about this issue -> https://github.com/paulmillr/chokidar/pull/1083


multinomial sampling should rely on binomial

addresses #183, but does not really fix it since binomial is naive sampling.
if we rely on binomial sampling then we can optimize once. This also updates the testing_boiler to work with Multinomial, notice how the tests are written to all use dynamically sized vectors and to rely on norms for approx absolute equality.
There is also a stoachastic test added, but I'm doubtful that I'm correct about both the test and sampling because it doesn't pass often enough. (Perhaps my choice of parameters in the test is not convergent enough to a normal distribution?) But a second pair of eyes on this would be helpful.

perhaps I need test statistics to assess if the samples are likely from the distribution...

notice how the tests are written to all use dynamically sized vectors

Hm... the static types (i.e. SMatrix, SVector) have more type safety and so I generally agree that the dynamically sized types need more testing. But I'm not sure about just testing the dynamic types; we might miss problems with specific nalgebra code paths or even incorrect trait bounds that would exclude the static types from being used in the statrs API altogether.
Looking at #165, this would also exclude basically all multivariate tests from no_std environments without an allocator.
Not 100% sure the existing boiler macro can cover everything... the nalgebra-based distributions are a fair bit different after all. But I think that e.g. new_from_nalgebra should be preferred over just new since it works for both types of Matrix/vector.

I agree we should be testing our API size generics and that tests should work on no_std if we have it. I'll revert that.
Do you have any other ideas to verify that samples are correctly generated?

Do you have any other ideas to verify that samples are correctly generated?

Not really, to be honest. Except for very rudimentary testing (samples are in the correct domain/range of values), you'd probably need to implement proper statistical test methods..

I'll rely on other implementations of statistical tests and rv sampling to verify my implementation for now. Once we've got tests and verification for them, then I'll consider bootstrapping features.

Tests use both static and dynamic allocators, but aren't broken out into separate tests, but they aren't interleaved into the test suite, so shouldn't be a big lift to conditionally compile them for a no_std option.

Certainly not a thorough analysis, but it's safe to say the implementation is not likely correct.
>>> stats.cramervonmises_2samp(scipy_mnom_gen(), data) # scipy vs data
CramerVonMisesResult(statistic=[4.46216824 4.365037   4.57871893 4.39759045 4.64458355], pvalue=[4.92114127e-11 7.65537633e-11 3.194799
98e-11 6.57013333e-11
 2.73471246e-11])
>>> stats.cramervonmises_2samp(scipy_mnom_gen(), scipy_mnom_gen()) # scipy vs scipy
CramerVonMisesResult(statistic=[0.1209659  0.05345861 0.15851277 0.10590302 0.13809556], pvalue=[0.49155904 0.8550833  0.36467217 0.556
74849 0.42800996])

tested with n=1000, p=5*[0.2] where both scipy and statrs generated 10k samples


Instance: mouse atlas check

Cloud Computing Instance Flavor
g3.xl - GPU instance (32 CPUs, 125 GB RAM and A100 GPU)
Description
download mouse atlas
ORCID
1234-5678-1234-5678
Email
U2FsdGVkX1+j3HfFeW96OHGOI232En/pUrIZNoK4vE8=
Confirm Email
U2FsdGVkX1+j3HfFeW96OHGOI232En/pUrIZNoK4vE8=

/create

/unshelve

/unshelve

/unshelve

/unshelve

/delete_instance

/create

/delete_instance

/delete_instance

/create

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/delete_instance

/create

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/delete_instance

/create

/delete_instance

/create

/delete_instance

/create

/delete_all

/create

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/shelve

/unshelve

/unshelve

/unshelve

/unshelve

/shelve

/unshelve

/unshelve

Following up @muratmaga comment indicating that the instance was automatically shelved after ~6 hours instead of the expected ~4 hours, here are some notes to help understand the underlying issue:

Instance unshelved on Dec 6th, 2024 at 4:19pm EST

Instance automatically shelved on Dec 6th, 20924 at 9:47pm EST

See 9:38pm log of the automatic shelving workflow leading to the change of state from active to shelved:
[...]
instance_name [instance-89] status [SHELVED_OFFLOADED] task_state []
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [5.33] -> action[shelve]
instance_name [instance-39] status [SHELVED_OFFLOADED] task_state []
instance_name [instance-64] status [SHELVED_OFFLOADED] task_state []
[...]

Automatic Instance Shelving
To help understand the timing, here are the logs of all the run of the automatic shelving workflow:

6:38pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [2.34] -> action[]
[...]

6:42pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [2.40] -> action[]
[...]

6:48pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [2.49] -> action[]
[...]

6:53pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [2.59] -> action[]
[...]

6:58pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [2.67] -> action[]
[...]

8:09pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [3.85] -> action[notify]
[...]

9:38pm log
[...]
instance_name [instance-62] status [ACTIVE] task_state []
issue_number [62]
instance_ip [<IP_ADDRESS>]
instance_name [instance-62], has_check_instance_shelve_script [yes], uptime_hours [5.33] -> action[shelve]
[...]

Workflow stats collected using the following:
gh run list -L 800 \
  --json workflowName,event,workflowDatabaseId,startedAt,updatedAt \
  --jq '[.[] | select((.startedAt | fromdateiso8601) >= ("2024-12-06T21:38:00Z" | fromdateiso8601)
      and (.startedAt | fromdateiso8601) <= ("2024-12-07T03:38:00Z" | fromdateiso8601))]' > ~/Projects/jupyter-notebooks/2024.12.07_MorphoCloudInstances-workflow-run-stats.json

... and then plotted using the following:
Workflow Start and End Times
import json
import pandas as pd
import matplotlib.pyplot as plt
from pytz import timezone

# Load JSON data
with open(input_data, "r") as file:
    workflows = json.load(file)

# Convert to DataFrame
df = pd.DataFrame(workflows)

# Parse timestamps and convert them to ET timezone
et = timezone('US/Eastern')

df['startedAt'] = pd.to_datetime(df['startedAt']).dt.tz_convert(et)
df['updatedAt'] = pd.to_datetime(df['updatedAt']).dt.tz_convert(et)

# Sort workflows by start time for better visualization
df = df.sort_values(by='startedAt')

# Create a plot
plt.figure(figsize=(12, 8))

# Plot start and end times for each workflow
for _, row in df.iterrows():
    plt.plot(
        [row['startedAt'], row['updatedAt']],
        [row['workflowName'], row['workflowName']],
        marker='o',
        linestyle='-',
        label=row['workflowName']
    )

# Customization
plt.title("Workflow Start and End Times (ET)")
plt.xlabel("Time (ET)")
plt.ylabel("Workflow Name")
plt.grid(True)
plt.xticks(rotation=45)
plt.tight_layout()

# Place the legend outside the plot for better clarity
plt.legend(loc="upper left", bbox_to_anchor=(1, 1))

# Show the plot
plt.show()

Workflow Execution and Delays for 'Automatic Instance Shelving'
import json
import pandas as pd
import matplotlib.pyplot as plt
from pytz import timezone

# Load filtered data
with open(input_data, "r") as file:
    workflows = json.load(file)

# Convert to DataFrame
df = pd.DataFrame(workflows)

# Parse timestamps and convert to ET
et = timezone('US/Eastern')

df['startedAt'] = pd.to_datetime(df['startedAt']).dt.tz_convert(et)
df['updatedAt'] = pd.to_datetime(df['updatedAt']).dt.tz_convert(et)

# Add runtime for each workflow
df['runtime'] = (df['updatedAt'] - df['startedAt']).dt.total_seconds()

# Filter "Automatic Instance Shelving"
shelving_df = df[df['workflowName'] == "Automatic Instance Shelving"]
preceding_df = df[df['workflowName'] != "Automatic Instance Shelving"]

# Plot all workflows with delays
plt.figure(figsize=(12, 8))

for _, shelving_row in shelving_df.iterrows():
    # Highlight the specific "Automatic Instance Shelving" workflow
    plt.axvline(shelving_row['startedAt'], color='red', linestyle='--', label=f"Shelving Start: {shelving_row['startedAt']}")

    # Identify overlapping workflows
    overlapping_workflows = preceding_df[
        (preceding_df['updatedAt'] >= shelving_row['startedAt'] - pd.Timedelta(hours=3)) &
        (preceding_df['startedAt'] <= shelving_row['startedAt'])
    ]

    # Plot preceding workflows
    for _, row in overlapping_workflows.iterrows():
        plt.plot(
            [row['startedAt'], row['updatedAt']],
            [row['workflowName'], row['workflowName']],
            marker='o', linestyle='-', label=f"{row['workflowName']} ({row['runtime']}s)"
        )

# Customization
plt.title("Workflow Execution and Delays for 'Automatic Instance Shelving'")
plt.xlabel("Time (ET)")
plt.ylabel("Workflow Name")
plt.grid(True)
plt.legend(bbox_to_anchor=(0.5, -1.7), loc="lower left")
plt.tight_layout()

# Show plot
plt.show()

Conclusion
Since the workflow executed prior the automatic shelving didn't take longer than expected, the delay in automatic shelving were likely introduced by delay in the scheduling of GitHub workflow.

Control Instance
9:38pm
12209204315

Automatic Instance Shelving
9:37pm
12209202578

Collect Instance Uptime
9:17pm
12209048823

Update Request Status Label
8:12pm
12208577722

Automatic Instance Shelving
8:09pm
12208552595

Collect Instance Uptime
7:48pm
12208387676

Update Request Status Label
6:58pm
12207919545

/unshelve

/unshelve

/restart

/unshelve

/unshelve

/unshelve

/unshelve

/delete_instance

/create

/email

/delete_instance

/create

/unshelve

/email

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/unshelve

/email

/unshelve

/unshelve


Example for validating a certificate chain

@peculiar/x509 is an easy to use TypeScript/Javascript library based on @peculiar/asn1-schema that makes generating X.509 Certificates and Certificate Requests as well as validating certificate chains easy.

I would like to check whether a given x.509 certificate has been signed by a known root certificate (CA). Can this library be used for that?

Yes
https://github.com/PeculiarVentures/x509#build-a-certificate-chain

Current version doesn't check certificate revocations.
You can use current API for path building. And check by yourself that the last cert in chain is trusted
const chain = new x509.X509ChainBuilder({
  certificates: certs,
});
const items = await chain.build(cert);

We are going to extend current API for chain validation described in RFC5280

Could use some better examples of this, ideally a simple 2 layer example.

Example
import * as x509 from "@peculiar/x509";

// Read certificates
const rootCert = new x509.X509Certificate(rootRaw);
const ca1Cert = new x509.X509Certificate(ca1Raw);
const ca2Cert = new x509.X509Certificate(ca2Raw);
const ca3Cert = new x509.X509Certificate(ca3Raw);
// ...
const leafCert = new x509.X509Certificate(leafRaw);

// Build chain
const certificates = [ca3Cert, ca2Cert, ca1Cert, /* ... */, rootCert];
const chain = new x509.X509ChainBuilder({
  certificates,
});
const items = await chain.build(leafCert);

// Print chain
for (const cert of items) {
  console.log(cert.subject);
}
console.log(items.toString("pem-chain"));

Output
CN=Client #1
CN=Intermediate CA #1.1.1
CN=Intermediate CA #1.1
CN=Intermediate CA #1
CN=Root CA

-----BEGIN CERTIFICATE-----
MIIC8jCCAdqgAwIBAgIBATANBgkqhkiG9w0BAQsFADAhMR8wHQYDVQQDExZJbnRl
...
5L4AqKQK14RU+4lFO5qhlaVSQd0PbWZoE1VOQG/5Chi8zxgMzus=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIC/TCCAeWgAwIBAgIBATANBgkqhkiG9w0BAQsFADAfMR0wGwYDVQQDExRJbnRl
...
76pvUKAWXKUUPCebfTawHY9q1ASQEsnCIHtQ4/WFlSdFbns2vxrKR1y5EpNanpn7
SA==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIC+TCCAeGgAwIBAgIBATANBgkqhkiG9w0BAQsFADAdMRswGQYDVQQDExJJbnRl
...
LsK2eGf1WtuvsrNUjmBie9/N+KpClRycBl2uRnOJMB/hb2IYJJXVIu8xsQJL
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIC7DCCAdSgAwIBAgIBATANBgkqhkiG9w0BAQsFADASMRAwDgYDVQQDEwdSb290
...
NEubGPJVpBz7zftQ1SbxWPjTXYF2f6QdwpPZ1wwPigg=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIICwDCCAaigAwIBAgIBATANBgkqhkiG9w0BAQsFADASMRAwDgYDVQQDEwdSb290
...
CtFu2HdSv7/M1NcNnueecn6B4YUCY4mueTXVsV9JJKM1T8XU
-----END CERTIFICATE-----

Thank you!
I was able to figure it out after I left this comment, but your code is probably better for future readers than mine would have been.
I suggest closing this issue.


add doc histories, serve release data, improve docs footer

Apologies for this giant PR, which snowballed into a bunch of things. ☃
Here are the highlights:
Electron Releases Data

Is now a public endpoint: https://electron-website-pr-960.herokuapp.com/releases.json
Includes everything that the GitHub Releases API returns
Now includes all (GitHub) releases, not just the last 100
Folds in data from https://atom.io/download/electron/index.json like v8 version, node, chromium, etc in a prop called dependencyVersions (open to alternative names for that prop)
Is 5.6MB (is that a problem?)

@Kilian I think this fixes #820
Once we land this, we can officially recommend it as a stable alternative to https://atom.io/download/electron/index.json.
Doc Histories
Every doc now has a page that lists all the versions of Electron that include it. Just add /history to any docs page, e.g. https://electron-website-pr-960.herokuapp.com/docs/api/browser-window/history
Here's what it looks like:

Docs Footer
Before:

After:

Tests should be passing now. This is ready for 👀

@vanessayuenn what do you think about the /docs/api/foo?history URL style?
Would it be better as /docs/api/foo/history ?

@zeke I was just thinking that! Yea I think /docs/api/boo/history makes more sense since it's a completely different view from /docs/api/foo.

Okay I changed the URL structure to /history instead of ?history, and updated the original post above accordingly.
Example: https://electron-website-pr-960.herokuapp.com/docs/api/touch-bar/history

https://electron-website-pr-960.herokuapp.com/docs/api/touch-bar/history

This is interesting, the page says the API was introduced in 1.6.3 but only has docs links for 1.6.5 and onwards? 😕 Is there something going wrong there?

@MarshallOfSound good eye. I'll investigate.

Electron release data:

the size is not a problem, for me at least. I do have a cronjob running that fetches this file every day though, so that's ~170MB a month that file will take in bandwidth from me alone. For electron-to-chromium I really only need the depencencyVersions bit, which is much smaller. To save bandwidth, maybe that can be split off separately?
This file doesn't seem to contain any of the beta versions. The old file does, for the most recent 1.8.2-beta.3 version at least. Is that a conscious choice? Using the beta designation we can support the 'unreleased' tag in browserslist.

@Kilian I've updated this PR to use the new electron-releases module I just wrote. Find out all about that at https://github.com/electron/electron-releases#readme
I've added a feature to the /releases.json endpoint that lets you pick the specific props you want from each object using the comma-delimited pick query param:

/releases.json gives you everything.
/releases.json?pick=tag_name,released_at,dependencyVersions.node gives you a lighter feed. Notice the . dot which you can use to pluck deep properties if needed.

What do you think?

That pretty nice, and now also included the beta versions, perfect.
There's something iffy though, https://electron-website-pr-960.herokuapp.com/releases.json?pick=dependencyVersions.chrome,dependencyVersions.version has a couple of empty objects and looking at the full file, those are the previous beta versions of 1.8.2, which are missing the dependencyVersions key. Is that on purpose?

@zeke probably cleaner that way indeed. As of https://github.com/Kilian/electron-to-chromium/releases/tag/v1.3.30 I now use electron-releases :)

@Kilian 👍


Add published with new draft state

Trello: https://trello.com/c/8PzlTDxX
Run publishing_e2e_tests against https://github.com/alphagov/publishing-e2e-tests/pull/244
Motivation
Now that the draft edition can be independently maintained, the properties of the draft and published edition can be different. Currently, if there is a published and draft edition, only the details for the draft taxon are displayed, so it is hard to tell if a published version already exists.
Changes
Before

After
New state "published with new draft"

Shows links to "preview" and "published" topic pages

Existing state "published"

Only shows link to published topic page

Existing state "draft"

Only shows link to preview topic page

@koetsier @cbaines I've addressed your comments and fixed up the code. Cheers!


[WIP] Rewrite @ScalaSignature when shading

So shaded artifacts can be used as libraries. See https://github.com/sbt/sbt-assembly/issues/350
This adds an extra ScalaSigProcessor step to the current jarjar steps. I copied MainProcessor into JJProcessor and scalaified it to be able to add the extra step.
This is still a WIP. We plan to work on the in our next sprint. I made this PR to gather feedback
on the approach.
There are some things that need to be done:

Support @ScalaLongSignature
Add tests (scripted + scalatest)
Idea: limitation on the rename rules that are supported. For example a new rule: renamePackage
Reason it i find it difficult to oversee which errors may get introduced by supporting the rename   rules as they are right now.
Support multiple scala versions. I currently only tested 2.12. Maybe we cannot use PickleBuffer the way we do now.
Cleanup, split ScalaSigAnnotationProcessor into multiple class files.

I could really use some insight of a scala compiler developer to check if I've missed some obvious tags to rename.

Assuming shading is needed by tools other than sbt-assembly, I wonder if it might be useful to split this as a library on its own. When it starts supporting library, there wouldn't be a need to shade only at the end of fat JAR? If so shading could potentially stand on its own as a plugin (we can continue to provide integration from sbt-assembly since it seems to be a popular feature), or even a CLI tool.

How about calling it Jar Jar Abrams?

@eed3si9n About creating a new library (a can't reply to your comment, maybe we should move discussion to the issue?)
We have only limited time to work on this so i was hoping to continue this as a part of sbt-assembly for now if you don't mind. We'll keep in mind that we want to extract in the future, once it gets some real life testing.
You're right we are not limited to fat jars anymore when this is done. I used sbt-assembly to publish a shaded, fat, library in a test project to test this so currently it already fits that use case. I do like the idea of an sbt-shade plugin but i don't think we have time to build that now (and we do like to start using the shaded jars internally asap)
Something else:
I did run into an issue with ivy though which does not elide all the shaded dependencies. I actually don't know if ivy dependencies can be excluded for an additional artifact with classifier. Do you?

I did run into an issue with ivy though which does not elide all the shaded dependencies. I actually don't know if ivy dependencies can be excluded for an additional artifact with classifier. Do you?

I have a FAQ entry here that might be relevant - https://github.com/sbt/sbt-assembly#q-despite-the-concerned-friends-i-still-want-publish-fat-jars-what-advice-do-you-have

Hey @eed3si9n , can you take a look at the PR?
I implemented most of the things from the initial todo list. I skipped .sig file support as i am not really sure this is actually useful in this plugin (and also how i'd test it)

Also I'm assuming you consent to releasing with the licensing terms of sbt-assembly? (MIT License)
I am asking this here since as noted above we might want to pull apart shading feature as a stand-alone library.

Sure! I consent to the licensing terms. Thanks for the review!

I've split this into a new library - http://eed3si9n.com/jarjar-abrams

@eed3si9n
Cool! Are you willing to add my company name (Simacan) to the blog post as I got to do most of the work in office hours?

Absolutely. I just added a line.


The C# Example on "Send a sharing invitation" page does not work

As per the title, the C# Example on the "Send a sharing invitation" page does not work.
Specifically, it fails with a 400 Error.
According to the "Microsoft Graph error responses and resource types" page, this signifies "Can't process the request because it's malformed or incorrect."
Therefore I request that this page please be updated, so that users can see the correct way to send Invitations.
In case it helps with diagnosing this issue, I see the below warning in Visual Studio when I add this snippet:
[] (local variable) GraphServiceClient graphServiceClient
CS0618: 'InviteRequestBuilder.PostAsync(Invite PostRequestBody,
Action<InviteRequestBuilder.InviteRequestBuilder PostRequestConfiguration>?, Cancellation Token)' is obsolete: 'This method is obsolete. Use PostAsInvitePostResponse instead.'

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b75e1562-24d6-bfaf-be46-d52ee93ffb1d
Version Independent ID: cd35f8f6-774e-70d3-9bd5-11110ab450b1
Content: Send an invite to access an item - Microsoft Graph v1.0
Content Source: api-reference/v1.0/api/driveitem-invite.md
Product: sharepoint
Technology: microsoft-graph
GitHub Login: @spgraph-docs-team
Microsoft Alias: MSGraphDocsVteam

I figured out why this didn't work. In the sample, there is no $ symbol before each the opening quotation mark in the final line of code. Consequently, it does not use the values of the drive-id and driveItem-id variables, but instead their literal names.
Therefore, I edited the last line as shown below, by adding the $ symbols. I just wanted to mention this in case it helps anyone else who encounters this issue.
var result = await graphClient.Drives[$"{drive-id}"].Items[$"{driveItem-id}"].Invite.PostAsInvitePostResponseAsync(requestBody);


backport: merge bitcoin#21404, #21415, #19314, #21882, #16939, #19084, #20602, #20253, #20480, #20736 (remove deprecated logic, use standard library alternatives)

this conflicts with https://github.com/dashpay/dash/pull/5051, but trivially rebased on top of it. Going to review 5051 first

I've rebased this on top of develop, and resolved a trivial include conflict


Not able to start Envoy with demo configuration in latest code.

Title: Getting error while launching envoy with demo configuration invalid value Invalid type URL, unknown type: envoy.extensions.access_loggers.stream.v3
Description: Trying from below url
https://www.envoyproxy.io/docs/envoy/latest/start/quick-start/run-envoy#run-envoy-with-the-demo-configuration
Error details:
Unable to parse JSON as proto (INVALID_ARGUMENT:(access_log[0].typed_config): invalid value Invalid type URL, unknown type: envoy.extensions.access_loggers.stream.v3.StdoutAccessLog for type Any): {"static_resources":{"listeners":[{"address":{"socket_address":{"address":"<IP_ADDRESS>","port_value":10000}},"name":"listener_0","filter_chains":[{"filters":[{"typed_config":{"http_filters":[{"name":"envoy.filters.http.router"}],"access_log":[{"name":"envoy.access_loggers.stdout","typed_config":{"@type":"type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog"}}],"@type":"type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager","route_config":{"name":"local_route","virtual_hosts":[{"routes":[{"match":{"prefix":"/"},"route":{"host_rewrite_literal":"www.envoyproxy.io","cluster":"service_envoyproxy_io"}}],"name":"local_service","domains":["*"]}]},"stat_prefix":"ingress_http"},"name":"envoy.filters.network.http_connection_manager"}]}]}],"clusters":[{"type":"LOGICAL_DNS","connect_timeout":"30s","dns_lookup_family":"V4_ONLY","transport_socket":{"typed_config":{"sni":"www.envoyproxy.io","@type":"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext"},"name":"envoy.transport_sockets.tls"},"name":"service_envoyproxy_io","load_assignment":{"endpoints":[{"lb_endpoints":[{"endpoint":{"address":{"socket_address":{"port_value":443,"address":"www.envoyproxy.io"}}}}]}],"cluster_name":"service_envoyproxy_io"}}]}}

hi @ankatare this suggests a mismatch between the version of envoy that you are using and the docs you are following
could you either ensure you are using envoy built on latest code or check the docs for the envoy version you are using

@phlax tahnks for quick response. code was on 17.1. on latest 1.18.2 it is fine.

May be we can close this...

yep - i think so - not sure how we can flag this better - one idea i had is using versionadded directives (see #15763 ) but not sure how easy or whether it will resolve this

@phlax one more way is to use --mode validate with each configuration before to start envoy :) . WDYT ?

yep - i think so - not sure how we can flag this better - one idea i had is using versionadded directives (see #15763 ) but not sure how easy or whether it will resolve this

yes... it is good idea

Closing this incidence as running good in latest code.

This is still an issue with all your latest code.

hi @learnbyseven please reread my comment here https://github.com/envoyproxy/envoy/issues/16475#issuecomment-840344681

Hi @phlax , I have just started with envoy following installation guide (Latest one) , My envoy version shows; /1.18.2/clean-getenvoy-76c310e-envoy/RELEASE/BoringSSL and I have downloaded and use the latest sample from https://www.envoyproxy.io/docs/envoy/latest/_downloads/92dcb9714fb6bc288d042029b34c0de4/envoy-demo.yaml file. I solved it just with by using putting a "connect_timeout: 5s".
Example:
clusters:

name: service_envoyproxy_io
type: LOGICAL_DNS
Comment out the following line to test on v6 networks
dns_lookup_family: V4_ONLY
connect_timeout: 5s

now , validate gives configuration 'envoy-demo.yaml' OK and envoy is running fine. Kindly review it one more time please.

with versions < 1.19 the connect_timeout setting is required, for later versions it has a default

I assume envoy package version has a coupling with OS version.
OS : DISTRIB_ID=Ubuntu, DISTRIB_RELEASE=18.04
Envoy : getenvoy-envoy/bionic,now 1.18.2.p0.gd362e79-1p75.g76c310e amd64 [installed]

its not so much related to the OS (atho its a factor) - it deps on what version you have installed
we are currently transitioning to a new packaging system, hopefully things will be clearer once that has been set up

Good to hear , I have updated the UB 18.04 with latest packages and install Envoy as per the latest doc (yesterday only) and got into the issue. No worries thanks ! for your swift responses, really appreciated.


recrawlableResolver with pre-v2.5.0 crawls in MongoDB

I'd really like to apply recrawlableResolver  with pre-v2.5.0 crawls registered in MongoDB. Is it possible to define the new 2.5.0 fields in MongoDB (and perhaps assign them some initial value to mimic the crawler), so the crawler can operate with them?.
Could you give me some guidance on that matter?
Thanks
Carlos

When it comes to MongoDB, I do not think the two new fields added (contentType and crawlDate), need to be explicitly added beforehand.  You may want to make a backup of your MongoDB if you are uncertain, but I suspect it will just work, except that each of your document will have to be recrawled at least once before they have the new values added (you will not see the benefits right away).
You can probably add default values manually by inserting values into Mongo yourself, but I am not sure how realistic that is for you and if it is really worth it (vs waiting one extra cycle until they get populated normally).
Please share your findings.

After downloding v2.5.0-SNAPSHOT, I have these files:
httpmime-4.2.6.jar    24/02/2016 12:20
httpmime-4.4.1.jar    01/04/2015 13:02

jempbox-1.8.11.jar                      24/02/2016 12:20
jempbox-2.0.0-20140823.120514-532.jar   15/06/2015 22:52

Which ones Should I keep?

I always keep the latest version of all jars to prevent issues.

@csaezl, are things OK for you now?  Can this ticket be closed?

Yes, you can close it. I'll come back to it later


Merge IISIntegration into release/2.1

Items remaining:

[ ] Get CI working (IISIntegration building on each PR)
[ ] Remove IISIntegration submodule from mondo
[x] Add back gtest submodule (removed due to bad merge)

@natemcmaster gonna go ahead with merging this. This will not affect update universe/ submodules at all, simply porting source code.


Add ability to disable filename sanitation

In #193 I noted that you do a few things to the filename here.
I've since experimentally disabled any sanitation in a local dockerfile, by just returning the input in that method. And it worked well for my purpose. Not only diesnt it cut off the filename, all the naming remains (e.g. : in a filename).
I'm uncertain why you did this, but if we would be given an option to disable this then I'd appreciate it!

Hi @9SMTM6. Some OS (e.g. Windows) do not allow characters like : and too long filename.
If there's no safe_filename(), it sometimes causes an error when transcribing via Youtube API because of the title.
Of course, this most likely happens when you're only transcribing from Youtube.
I think it's probably safe to disable this when transcribing a local file, because it probably already has a safe filename.

@jhj0517 thanks for the info.
Blanko disabling it does seem to cause some issues with certain filename. It probably executes unintended codepaths. I believe the trigger was multiple dots in the filename. I'll have to come back to you for that, just wanted to let you know.

Thanks for bringing this up.
Since gradio already does the filename sanitization, I don't think I need to do it twice.
safe_filename() is disabled when transcribing a local file in #234.
Please feel free to reopen the issue if you encounter any problems!


Allow Chat Endpoint to modify Database Models

Fixed by #21, further issues should be opened for more specific model modifications


Use composite in the calling sample

What
Replace all the sample functionality that is a duplicate of the composite functionality with the composite itself
Why
We can't test both the samples and composites - we need one hot path for testing.
This will also make sure our composites are being flexible for real app usage.
How Tested

Started new Call
Shared link and joined in another tab
Left call - showed call ended screen
Rejoined call
Shows error screen if server failed to return user token
Joined teams meeting
tbd:
kicked from teams meeting

Excellent, that solves a lot of current problems we have for now! Do we have 2-3 screenshot to show what will be the visual difference between the non-composite one and composite one?

Here's some. The reason I didn't add to the PR description was I don't want these reviewed, whether we like the different looks or not will be filed as composite improvement bugs but not tackled in this PR.

Before
After


When posting from Twitter to Mastodon convert @xx in<EMAIL_ADDRESS>

I think it could be a plus for mastodon user to see that the post come from twitter
so they don't look for the @xx on their instance and understand that it is from twitter
The idea is to add a conversion in twitter post changing every @xx in<EMAIL_ADDRESS>I don't know much python but if you are ok but don't have time to do it I can try a PR
Have a good day

Oh, I've started #38 already to handle this as part of a few other changes I've wanted to add, let's see if it gets accepted, though. Ideally, it should work the other way around too, but Twitter's tighter character limit makes it tougher to signal a given mention comes from Mastodon instead of Twitter.

Closed by PR.


bug(python): creating a table with a generator function gives strange error if no schema specified

LanceDB version
v0.8.1
What happened?
Using the async python code I tried to create a table using a generator but forgot to specify the schema.  The error I received was:
Traceback (most recent call last):
  File "/home/pace/dev/lance-experiments/v2-beta/mass_ingest.py", line 44, in <module>
    asyncio.run(main())
  File "/home/pace/miniconda3/envs/lancedb/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/pace/miniconda3/envs/lancedb/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pace/miniconda3/envs/lancedb/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/pace/dev/lance-experiments/v2-beta/mass_ingest.py", line 38, in main
    await db.create_table("images", load_images(batch_size))
  File "/home/pace/dev/lancedb/python/python/lancedb/db.py", line 758, in create_table
    validate_schema(schema)
  File "/home/pace/dev/lancedb/python/python/lancedb/common.py", line 116, in validate_schema
    if schema.metadata is not None:
       ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'metadata'

This is a non-obvious error
Are there known steps to reproduce?
# Haven't tested this but it should be roughly correct
import lancedb
import pyarrow as pa

async def main():
    db = await lancedb.connect_async("/tmp/foo")
    def make_data():
        yield pa.table({"x": [1, 2, 3]})
    db.create_table("foo", make_data())

Verified, this is not an issue anymore.


Added Kayoanime website

Related Issue
Closes #756

Description
Added kayoanime website
Screenshots

Checklist

[x] My code adheres to the established style guidelines of the project.
[x] I have included comments in areas that may be difficult to understand.
[x] My changes have not introduced any new warnings.
[x] I have conducted a self-review of my code.

add a shorten URL for the image

use url shortner for Image URL

Hello, contributors! This is Jason speaking. Due to some workflow issues, we will be closing all the PRs today. Therefore, I kindly request all of you to create new PRs tomorrow, as the currently open ones will be closed by our team. I apologize for any inconvenience caused. Also, please make sure to check the titles when creating the PRs. You can refer to the README.md for more details. Thank you.


Diverging forks

This is more of a meta-issue, but for lack of a better idea of where to post it, I'll do so here.
@Lonami It's good to see you back active again.
As I mentioned in another thread, I've done a lot of work implementing vectorization, and have it near working.  I have just a few remaining issues about using sensors to update vectors at the right time, auto type-detection for edge cases like never-assigned variables, and cleaning away clutter that ended up not being necessary.  Automatic type detection, vectorization, and a robust local/global system required reworking a lot of the inner representation of Instructions and of variables/atoms, making the latter their own data-types organized in a class hierarchy, rather than just cramming them into big strings like our old code did.
As a consequence, any change you make to the old way of doing things, like the commits you recently made, will end up needing some work if it's going to be translated into the new Atom-based system.  That work may be tricky for you, at least until you figure out the new system.  For me, it'd generally be about as easy to do these small commits from scratch as it would be to try to convert from yours.
I'm not sure what the best solution to all this is.  It may be that you'll decide you aren't interested in taking on board all my reworked vectorized stuff, in which case we'll just end up with two separate forks.  Or it may be that one of us will end up reconciling the two together.  If that's the route we end up going, I think the easiest path, in the next few days, may be just for us to agree in the discussion about what the syntax should be, and have me incorporate that directly, rather than having our forks continue to diverge.  At any rate, when I post issues here, I'm usually thinking of them as decision points that you might want to have a say in, rather than tasks for you to do.  But of course you're welcome to do whatever you want...

Unfortunately, our development process is far too different for it to be worth it for me to keep track of all the changes. Basically, the cost of implementing some feature from scratch on my own is likely to be a lot lower than scanning through your huge commits and picking up the relevant parts, adding them in smaller commits in my code.
(Note: I'm not saying or implying that your development process is inherently bad, I'm simply pointing out that it's different from mine).
At the moment, I have no plan to unify both forks again, but if you ever want to do that once this main repository is feature-complete enough, you're more than welcome to do so. So, I think it's fine for the forks to have diverged.
I'm going to close this issue since it's not tracking any particular bug, but feel free to keep posting comments or discuss it further.

Unfortunately, our development process is far too different for it to be worth it for me to keep track of all the changes. Basically, the cost of implementing some feature from scratch on my own is likely to be a lot lower than scanning through your huge commits and picking up the relevant parts, adding them in smaller commits in my code.
(Note: I'm not saying or implying that your development process is inherently bad, I'm simply pointing out that it's different from mine).
At the moment, I have no plan to unify both forks again, but if you ever want to do that once this main repository is feature-complete enough, you're more than welcome to do so. So, I think it's fine for the forks to have diverged.
I'm going to close this issue since it's not tracking any particular bug, but feel free to keep posting comments or discuss it further.

If at some point one of us wants to re-merge these, here are the main steps that would be involved in converting an instruction created your old way (like the ones in your recent updates) to one created my new way.
A typical old way of creating an instruction would prepare atoms for use as arguments in an instruction via compiler.as_target() and compiler.as_value().  My current division of labor uses as_variable() to construe something as a variable (e.g. as a target for setting), and takes care to link multiple instances of the same variable together, which helps for type-tracking, local/global distinction, function substitutions, and vectorization.  My as_Atom() is the rough equivalent of the old as_value, generating an atomic way of referring to some AST-node, generating a dummy variable for it if necessary.  Since "Atoms" are the primary building blocks of Instructions in my new way of doing things, it makes more sense to call this "as_Atom".  As_atom ends up calling upon as_variable to deal with variables, to put a lot of common functionality into one place.
A typical old way of creating an instruction concatenated its arguments together into an f-string using something like f"command {arg1} {arg2} {arg3}".   In the new way of doing things, this would instead be Command_Ins( arg1, arg2, arg3 ) where Command_Ins is defined in my mlog.py and includes information about number and types of arguments and vectorizability for this type of instruction.
For most run-of-the-mill practical purposes, those are the main things that have changed, though adding new Instruction types (e.g., I still haven't added Unit.commands) or adding new types of Atoms (e.g., if we follow up on our idea of allowing a Stack data-structure) would require delving into the details of how Instructions and Atoms are implemented.  That'd be straightforward for me to do (mostly just copying the closest existing analog and making relevant tweaks), but it would perhaps be more involved than you'd want to do.

If at some point one of us wants to re-merge these, here are the main steps that would be involved in converting an instruction created your old way (like the ones in your recent updates) to one created my new way.
A typical old way of creating an instruction would prepare atoms for use as arguments in an instruction via compiler.as_target() and compiler.as_value().  My current division of labor uses as_variable() to construe something as a variable (e.g. as a target for setting), and takes care to link multiple instances of the same variable together, which helps for type-tracking, local/global distinction, function substitutions, and vectorization.  My as_Atom() is the rough equivalent of the old as_value, generating an atomic way of referring to some AST-node, generating a dummy variable for it if necessary.  Since "Atoms" are the primary building blocks of Instructions in my new way of doing things, it makes more sense to call this "as_Atom".  As_atom ends up calling upon as_variable to deal with variables, to put a lot of common functionality into one place.
A typical old way of creating an instruction concatenated its arguments together into an f-string using something like f"command {arg1} {arg2} {arg3}".   In the new way of doing things, this would instead be Command_Ins( arg1, arg2, arg3 ) where Command_Ins is defined in my mlog.py and includes information about number and types of arguments and vectorizability for this type of instruction.
For most run-of-the-mill practical purposes, those are the main things that have changed, though adding new Instruction types (e.g., I still haven't added Unit.commands) or adding new types of Atoms (e.g., if we follow up on our idea of allowing a Stack data-structure) would require delving into the details of how Instructions and Atoms are implemented.  That'd be straightforward for me to do (mostly just copying the closest existing analog and making relevant tweaks), but it would perhaps be more involved than you'd want to do.


Helm Install results in a CVAT instance where I am unable to login

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

Following the helm-chart install procedure produces a CVAT instance that is impossible to log in to:

Expected Behaviour

I would expect the superuser login described in How to crate a superuser or "Create an account" to successfully log into the CVAT annotation tool.
Current Behaviour

Instead, The "Could not check autorization on the sever" message was returned each time.  The error details on the browser console only returns "Not Allowed" which is not helpful in diagnosing the problem.
Possible Solution

Steps to Reproduce (for bugs)

Helm Install:
Helm Overrides:
$ cat helm-chart/values.override.yaml
postgresql:
  secret:
    password: cvat_passwd
    postgres_password: cvat_passwd
    replication_password: cvat_passwd

Helm Install from clone of CVAT repository
$ helm upgrade cvat --install ./helm-chart -f ./helm-chart/values.yaml -f ./helm-chart/values.override.yaml --namespace cv --create-namespace
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /var/snap/microk8s/2488/credentials/client.config
Release "cvat" does not exist. Installing it now.
NAME: cvat
LAST DEPLOYED: Fri Sep 24 14:27:00 2021
NAMESPACE: cv
STATUS: deployed
REVISION: 1
TEST SUITE: None

CVAT runtime following helm install:
$ kubectl get all -n cv
NAME                                 READY   STATUS    RESTARTS   AGE
pod/cvat-frontend-54674747d6-n8ccf   1/1     Running   0          65s
pod/cvat-backend-6f7d85f558-4tqnk    1/1     Running   0          65s
pod/cvat-postgresql-0                1/1     Running   0          65s
pod/cvat-redis-master-0              1/1     Running   0          65s

NAME                               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/cvat-redis-headless        ClusterIP   None             <none>        6379/TCP   66s
service/cvat-postgresql-headless   ClusterIP   None             <none>        5432/TCP   66s
service/cvat-postgresql            ClusterIP   <IP_ADDRESS>     <none>        5432/TCP   65s
service/cvat-redis-master          ClusterIP   <IP_ADDRESS>    <none>        6379/TCP   65s
service/cvat-frontend-service      ClusterIP   <IP_ADDRESS>   <none>        80/TCP     65s
service/cvat-backend-service       ClusterIP   <IP_ADDRESS>   <none>        8080/TCP   65s

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cvat-frontend   1/1     1            1           65s
deployment.apps/cvat-backend    1/1     1            1           65s

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/cvat-frontend-54674747d6   1         1         1       65s
replicaset.apps/cvat-backend-6f7d85f558    1         1         1       65s

NAME                                 READY   AGE
statefulset.apps/cvat-postgresql     1/1     65s
statefulset.apps/cvat-redis-master   1/1     65s

Port forward backend:
$ kubectl port-forward service/cvat-backend-service 8080:8080 -n cv
Forwarding from <IP_ADDRESS>:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Port forward frontend:
$ kubectl port-forward service/cvat-frontend-service 8880:80 -n cv
Forwarding from <IP_ADDRESS>:8880 -> 80

Chrome version: Chrome/94.0.4606.61 Safari/537.36" "-"
Context

Without the helm install, we cannot evaluate CVAT in our Kubernetes production environment.

Kubernetes
Operating System and version (e.g. Linux, Windows, MacOS):

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

Kubernetes and helm versions:
$ snap list microk8s
Name      Version  Rev   Tracking     Publisher   Notes
microk8s  v1.22.2  2488  1.22/stable  canonical✓  classic
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22+", GitVersion:"v1.22.2-3+a4bd0397e1cb5e", GitCommit:"a4bd0397e1cb5e638a6a6464f759114ee2250ea1", GitTreeState:"clean", BuildDate:"2021-09-16T00:05:14Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22+", GitVersion:"v1.22.2-3+a4bd0397e1cb5e", GitCommit:"a4bd0397e1cb5e638a6a6464f759114ee2250ea1", GitTreeState:"clean", BuildDate:"2021-09-16T00:00:16Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Helm:
$ helm version
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /var/snap/microk8s/2488/credentials/client.config
version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.6"}

Code example or link to GitHub repo or gist to reproduce problem:
Other diagnostic information / logs:
Logs from `cvat` container

Backend log:
$ kubectl logs deployment.apps/cvat-backend -n cv --tail=20
debug1: process_message: socket 1 (fd=4) type 11

2021-09-24 21:36:09,072 DEBG 'rqworker_default_0' stderr output:
DEBUG - 2021-09-24 21:36:09,072 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-09-24 21:36:09,073 DEBG 'rqworker_default_1' stderr output:
DEBUG - 2021-09-24 21:36:09,072 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-09-24 21:36:09,474 DEBG 'rqworker_low' stderr output:
DEBUG - 2021-09-24 21:36:09,474 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-09-24 21:42:54,091 DEBG 'rqworker_default_1' stderr output:
DEBUG - 2021-09-24 21:42:54,091 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-09-24 21:42:54,092 DEBG 'rqworker_default_0' stderr output:
DEBUG - 2021-09-24 21:42:54,091 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

2021-09-24 21:42:54,492 DEBG 'rqworker_low' stderr output:
DEBUG - 2021-09-24 21:42:54,491 - worker - Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.

Compare docker compose yaml and helm install to identify the missing communication

I've also been looking at this, but with an external postgres sql database.  I installed with an override file:
helm upgrade cvat --install . -f values.yaml -f values.override.yaml --namespace cvat-namespace

Containing enabled: false, user: postgres, password: mypassword, host=my_ip.
After install I can't login or create a super user using:
kubectl exec -it --namespace $HELM_RELEASE_NAMESPACE $BACKEND_POD_NAME -c cvat-backend-app-container -- python manage.py createsuperuser

I get errors around the database usage.
To understand I logged in and took a look around:
$ kubectl exec -it --namespace $HELM_RELEASE_NAMESPACE $BACKEND_POD_NAME -c cvat-backend-app-container -- bash
$ env | grep -i postgres
CVAT_POSTGRES_DBNAME=cvat
CVAT_POSTGRESQL_SERVICE_HOST=<IP_ADDRESS>
CVAT_POSTGRESQL_PORT_5432_TCP_PROTO=tcp
CVAT_POSTGRES_PASSWORD=cvat_postgresql
CVAT_POSTGRESQL_PORT_5432_TCP_PORT=5432
CVAT_POSTGRESQL_SERVICE_PORT=5432
CVAT_POSTGRES_USER=cvat
CVAT_POSTGRESQL_PORT=tcp://<IP_ADDRESS>:5432
CVAT_POSTGRES_HOST=cvat-postgresql
CVAT_POSTGRESQL_SERVICE_PORT_TCP_POSTGRESQL=5432
CVAT_POSTGRESQL_PORT_5432_TCP_ADDR=<IP_ADDRESS>
CVAT_POSTGRESQL_PORT_5432_TCP=tcp://<IP_ADDRESS>:5432

The password is not the one I set, ip address isn't the one I set, etc.  So I changed the environment variables from the terminal and was able to successfully create a user:
$ export CVAT_POSTGRES_USER=postgres
$ export CVAT_POSTGRES_HOST=my_ip
$ export CVAT_POSTGRESQL_SERVICE_HOST=my_ip
$ export CVAT_POSTGRESQL_PORT=tcp://my_ip5432
$ export CVAT_POSTGRESQL_PORT_5432_TCP_ADDR=my_ip
$ export CVAT_POSTGRESQL_PORT_5432_TCP=tcp://my_ip:5432
$ export CVAT_POSTGRES_PASSWORD=mypassword
$ python3 manage.py createsuperuser

I have been tinkering with the values.yml and values.override.yml files and haven't been able to see any change in my environment variables.

I found a bug related to using an external database and made a fix here: #3779
I am able to run createsuperuser successfully, however I'm still not able to log in with the account that I have created.
I checked the frontend app and noticed that it also has some environment variables related to the postgres database that are not using my settings in the values.override.yaml.  I suspect that is the cause of my login problem.
$ kubectl exec -it --namespace $HELM_RELEASE_NAMESPACE $FRONTEND_POD_NAME -c cvat-frontend-app-container -- sh
# env | grep -i postgres

I was experiencing the same issue last week. Similarly, I deployed CVAT on Kubernetes cluster with my modified deployment files. The main reason of this issue is about request routing on k8s cluster. Ingress on k8s is more complicated than the original docker-compose network.
From your logs in deployment.apps/cvat-frontend, you can see that a lot of request are routed to frontend. However:

only these should be routed to frontend pod: /cvat-ui.f9340a4e583f3306fd17.min.js, /favicon.ico
these should be routed to backend pod: /api/v1/auth/, /api/v1/users/, /auth/login, ...

I used ingress to configure routing. I see ingress is mentioned in helm chart but routing rules are not configured correctly.
Hope these can help you debug. Deploying CVAT on k8s without official support is painful.

FYI, I'm using CVAT 1.6.
Upstream CVAT version: 'release-1.6.0' 8d03ed7959632d6f3ffd76020ad577fa409acfcd

@TianbaoLi , thanks for comments and your suggestions.
@bhlarson , we are going to support helm charts officially in the nearest future. For now I will close the issue.

I was experiencing the same issue last week. Similarly, I deployed CVAT on Kubernetes cluster with my modified deployment files. The main reason of this issue is about request routing on k8s cluster. Ingress on k8s is more complicated than the original docker-compose network. From your logs in deployment.apps/cvat-frontend, you can see that a lot of request are routed to frontend. However:

only these should be routed to frontend pod: /cvat-ui.f9340a4e583f3306fd17.min.js, /favicon.ico
these should be routed to backend pod: /api/v1/auth/, /api/v1/users/, /auth/login, ...

I used ingress to configure routing. I see ingress is mentioned in helm chart but routing rules are not configured correctly. Hope these can help you debug. Deploying CVAT on k8s without official support is painful.

Thanks very much. I use this configuration in k8s and work
spec:
  rules:
    - host: myhost
      http:
        paths:
          - path: /
            backend:
              serviceName: cvat-ui
              servicePort: 80
          - path: /api
            backend:
              serviceName: cvat-server
              servicePort: 8080

if don't user ingress, only nodeport how should change routed


Update Cluster with rcu lock

clustermanager's update api should return success when no request/stream use old cluster config, or until timeout

fix by #334


Some fixes

Syntax fix for node <= 6 ;
Try catch for request.post

Please update at https://www.npmjs.com/package/node-qiwi-api and add github link to the package

Done


wrapper around window.fetch calls

For https://github.com/alpheios-project/components/issues/147 we need the ability to execute the lemma-client calls in node.js environment. The window.fetch method is not available there.
I would like to be able to keep using window.fetch when we have it, to avoid unnecessary runtime dependencies on 3rd party libraries, but to fallback to an alternative method, such as using the axios library, when window.fetch isn't available.

This was implemented with #5


Add record screen (with MediaProjection and virtualdisplay) source.

hi there , reacently , i try to record screen with MediaProjection and virtualdisplay to push to srt server , i am faceing some kind of issus, could you  please and the screen record feature in the futer release
Originally posted by @DesenYang in https://github.com/ThibaultBee/StreamPack/issues/14#issuecomment-873556059

@LionMiao moving your implementation here: https://github.com/thiagooo0/StreamPack When I developed StreamPack, I did not plan to use anything but the camera. So it might be a bit tricky to develop a streamer for MediaProject. I will definitely have a look on that but I won't have time before 2-3 weeks. I plan to create new streamers (one for SRT, one for file). What is your use case so the implementation is close to what you attend to do?

I want to make a remote desktop app between two android devices.


Child sessions that finish launching after their parent exits can break the debug UI

See https://github.com/microsoft/vscode-js-debug/issues/390#issuecomment-605147836: You should be able to reproduce by following the verification instructions with a js-debug-nightly version <= 2020.3.2517

I see you were deep debugging into this. Do you have a proposed solution? How should the VS Code UI behave?

Perhaps two things:

Double check the parent session after returning from initialize. Iff it's present and in State.Inactive, then terminate it and reject with a 'canceled' error

https://github.com/microsoft/vscode/blob/2d85caa55586161faad7ce7c54e7335d3f14ac97/src/vs/workbench/contrib/debug/browser/debugService.ts#L531

For the specific problem, it's probably safe to ignoreErrors in .expand. I see we do this elsewhere in the file. The set is cleared after that function runs, so ignoring problems here shouldn't have negative effects on state elsewhere.

https://github.com/microsoft/vscode/blob/52df5961f3e8bb669de34fd32ab1f9df25bec5e2/src/vs/workbench/contrib/debug/browser/callStackView.ts#L136

These two are great proposals. I can look into them, thus assigning to March.
If you want a PR is also welcome, if not I will tackle on Monday.


Add -C 16384 -O bigalloc to ext4 formatting for improved SSD performance

Updated ext4 formatting options to include -C 16384 and -O bigalloc.
Using a block size of 16384, aligned with the native page size of most modern SSDs, enhances performance and reduces write amplification.
This should close #770

Thanks @vexr.
Tagging @EmilFattakhov for visibility.


Feature Request - smart compression on upload with limited bandwidth

My ISP provides an asymmetric connection, as many other people have.  My upload bandwidth is only a tenth of my download bandwidth.
It would be WONDERFUL if AzureStorageExplorer provided some assistance with this.
The BEST incarnation of this would be to automatically detect upload speeds (and local processor speed), and silently compress the data stream at my end, uncompressing at the Azure end, so it's totally transparent and the compression/transmission is optimized for minimum time to upload.
However, that might be too hard / complicated / consuming of developer time.
A SIMPLER incarnation might be to offer an option where you select file(s) and choose a menu option "optimize upload", and it then:

Tests your upload speed (briefly)
Does some quick compression calculations involving the selected files
Compresses the files into ZIP using the optimal amount of compression to balance the time spent doing that against the time in transmission
Failing that, at least gives the user the settings to use for an Open Source ZIP or 7z compressor.

@cforbin1 we'll pass along this feedback to the service team, but unfortunately the main thing you're asking for ("compress the data stream at my end, uncompressing at the Azure end") is out of our scope. :(
With regards to upload speeds though, it sounds like you should try changing the Network Concurrency setting to "Adjust dynamically":

(Settings, aka, gear on the left toolbar > Transfers > AzCopy > Network Concurrency)
That mode will have AzCopy throttle and test itself more aggressively than normal.
You can also of course compress your files yourself and then upload them. But they will be compressed on the server and need to be decompressed after you download them. There's also a transfer setting for "Decompress Files":

So if you were to compress each of your files to .gz/.gzip/.zz, then on upload they should be given a content-type that indicates their compression. And when you download them, if that setting is enabled, AzCopy will decompress them.
Could you elaborate a bit more on what you mean by this though?

Failing that, at least gives the user the settings to use for an Open Source ZIP or 7z compressor.

Closing due to lack of response. If you'd like to give us more information about what you meant by

Failing that, at least gives the user the settings to use for an Open Source ZIP or 7z compressor.
Feel free to open a new issue.

If you'd like to talk to the AzCopy team about a better default value for their Network Concurrency setting, we'd recommend going to their GitHub and opening an issue there. Thanks.


not work Appended Routes

I did as shown in the documentation https://github.com/angular-ui/ui-router#nested-states--views - here is my code vieschs http://plnkr.co/edit/0DypwIajU7fOPvQGR5fg here it works.
But when I use it in my project template child is not called. Call always occurs only parent.
How can I find the problem?
My question is similar to this: http://stackoverflow.com/questions/22157357/angular-ui-ui-router-with-steroids
Clone issue http://stackoverflow.com/questions/24487382/angular-js-angular-ui-ui-router-not-work-appended-routes

https://gist.github.com/eddiemonge/f6a58169c2846731a1eb


k8s: ignore kubectl.kubernetes.io/last-applied-configuration annotation

This annotation does not give any real usability for CNP Status and it
might even increase the CNP status for a big policy with a small-medium
cluster (50 nodes).
Signed-off-by: André Martins<EMAIL_ADDRESS>

This change is

Coverage increased (+0.01%) to 43.138% when pulling 67da140bb685d460da0cdc278fe872277652e13a on pr/ignore-cnp-status-annotation into 0992a648380d0a9335c4e9093b83fc06622d7fba on master.

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please

hit https://github.com/cilium/cilium/issues/7125


benchmarking utilities

This PR resolves https://github.com/oasislabs/oasis-core/issues/2536.
Changelogs:
https://raw.githubusercontent.com/oasislabs/oasis-core/matevz/feature/bench/.changelog/2687.breaking.md
https://raw.githubusercontent.com/oasislabs/oasis-core/matevz/feature/bench/.changelog/2687.feature.md
https://raw.githubusercontent.com/oasislabs/oasis-core/matevz/feature/bench/.changelog/2687.internal.1.md
https://raw.githubusercontent.com/oasislabs/oasis-core/matevz/feature/bench/.changelog/2687.internal.2.md

sounds good as described

E2E tests seem to be failing on CI?

Also rebase on master.

@Yawning @ptrus since you also reviewed, can you take another quick look.

I think this looks good, just squash commits.

I just added some details to slack reporting. (e.g. link to build details, metrics thresholds env variable etc.)


No way to add annotations for Client methods

There's no way to add annotations for methods in the Client class as there isn't a template class. I created a PR ( #2616 ) that addresses this.

This issue was resolved with PR #2616 .


Enhancement Combine function produces incorrect value

Describe the bug
Using a Level 5 character, all enhancements had been "Plus'd out" to their maximum value of "2++".  With the Store option now available, I was able to purchase Level 5 enhancements, and attempt to combine them with the existing "2++".  When I did, it produced a value of "2+3", which should have been "5+"
To Reproduce
Have 2++ enhancements already in the slotted power.  Combine with Level 5 enhancement of the same type to produce the incorrect value.
Expected behavior
A clear and concise description of what you expected to happen.
Expected value would have been 5+
Screenshots or Attachments (if applicable)

I didn't even know the client could display +3

I had a 2+ that failed with a 5, and it decided to keep the 2+ instead of the 5.
I also added fives to 2+3 and got 2+4.


make sure provision resources in all samples using same tags

Purpose

Make sure provision resources in all samples using same tags.

Does this introduce a breaking change?
[ ] Yes
[x] No

Pull Request Type
What kind of change does this Pull Request introduce?
[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

Test the code

What to Check
Verify that the following are valid

...

Other Information

@chenrujun could you pls help to review this PR.
This PR is for Spring Cloud Azure 4.x version. If the PR merged, I will create a new PR to handle Spring Cloud Azure 6.x version.


Select/Deselect reference shaders based on platform/version being tested

For MSL, I've encountered a couple of use cases where being able to select and deselect reference shaders based on characteristics of the build...such as a platform type (iOS vs macOS)...or even a shader language version (MSL 2.0 vs 1.2).
Essentially...some MSL functions and parameters are not available in all MSL versions and on all platforms. Apple distinguishes between iOS and macOS...and MSL versions.
Another example is the use of SPIR-V optimizations (--opt). Optimizations of certain complex shaders sometimes create SPIR-V that is not straightforward to convert to MSL. Since we have no control over changes to SPIR-V optimizations...it would be useful to be able to selectively deselect those shaders from the optimization run.
What are our options for an approach here? Could we start to differentiate shader directories and selectively run them for platforms, versions, and optimizations from test_shaders.sh? Or could we think in terms of selecting shaders based on characteristics (eg- maybe a platform & version shader naming convention)?

PR #373 partially addresses this.
The MSL shaders that cannot be optimized have been moved to the new shaders-msl-no-opts folder.
MSL shaders in the shaders-msl are run both with and without optimizations. MSL shaders in the shaders-msl-no-opts folder are only run WITHOUT SPIR-V optimizations.
CompilerMSL now supports a platform (iOS or macOS) indicator to selectively output MSL according to platform requirements. This is in addition to the existing MSL version indication.
The ability to select this via the spirv-cross command line has NOT been implemented. Nor have any separate tests and reference shaders been developed for each separate platform...pending further discussion.

FWIW, I think this is resolved long ago.


Enable reading password from stdin, make prompt evident

I am currently using pass (a.k.a. password-store) together with a gpg-agent to store credentials.  This enables hamster-bridge to play nicely with such a store.
The second commit is a usability suggestion -- I think with current prompt format it is not immediately evident that user input is expected.

Nice, thanks!


404 error when recommender fails to find results

Either the recommender was unable to perform it's lookup or it did not find any results.
[2017-09-26 15:50:37,985] ERROR in views: Recommender failed (2018FrPhy..13.9801W): Unable to get results!
If the issue was executing the lookup, does this use postgres or solr?
If the problem is that there is no entry for 2018FrPhy..13.9801W can we please have this generate a 2XX http status code? Maybe the respone could be something like entry not found, but the 404 status code will eventually have a negative impact on user's reputation

Since this service is discontinued, I'm closing this issue.


[Storage] Relocated SAS generation

Can one of the admins verify this patch?

/azp run python - storage - tests

/azp run python - storage - ci


Freeze the Backbone and Dilated Encoder

Hi,
Many thanks for your awesome work. In the original Detectron2, we can freeze certain layers of the backbone with _C.MODEL.BACKBONE.FREEZE_AT.
Can you please let me know how to  freeze both the ResNet backbone AND the Dilated Encoder, and update only the Decoder during training in YOLOF as well, say using yolof_R_50_C5_1x ?

You can manually add the freeze codes for backbone and encoder here in the yolof file as we do in the darknet.


Windows compatibility

The debug_adapter currently relies on at least two features that are not available on Windows:

wait_for_input/3 on none-socket streams (stdin specifically)
Setting the encoding of the stdio streams - this may not actually be needed on Windows

The first problem can be solved by using an extra thread to wait on stdin and relay DAP messages to server via thread_send_message/2. In this solution, the server handler thread stops using wait_for_input/3 altogether and only handles messages from the thread message queue.
A draft for adding Windows compatibility can be found in https://github.com/eshelyaron/debug_adapter/pull/39, it currently just gets the tests running on Windows during CI.

Fixed in #42


Using Oracle Advanced Queuing with go-ora

I'm about to migrate a project (from godror to go-ora) that currently uses godror's NewQueue and Dequeue functions.
Do you have any plans to implement equivalent API functions in go-ora?
(I'm aware that I can already get what I need via PL/SQL, but native go-ora API functions would be really nice :-) ).

I will see code first and inform you

I make simple implementation for AQ in go-ora/dbms
example found in examples/dbms_aq
please see if it need more modification before release

In my projects I use go_ora.NewConnection rather than sql.Open, so it looks like I can't use the functions you've added so far in aq.go. Would you be able to extend the aq.go functions to also support direct use of the package?

I've been using go_ora.NewConnection only because the benefit here is that you can use pl/sql and output parameters.  However, I noticed that your Dequeue and Enqueue functions in aq.go use pl/sql and output parameters in conjunction with sql.Open.  Am I right that there's no longer any advantage to using go_ora.NewConnection instead of sql.Open?

sorry I am updating documentation
as the package advance the support for database/sql package increase
until now only feature that is not found in database/sql is query to struct using db tag which require direct package use

Hi! Thank you for such examples, but if i want to get for example 1000 events from AQ process them and if all ok - dequeue it, if something happens - able to read it again from aq?
for example - i';m read 1000 messages from aq and pass it to kafka, if producing successful i'm remove this messsages from aq, if something bad happens - service restarted and re-read again this messages from aq ?

@sijms can you share some info for me?


BMC file mirroring and ALT side switch testing

[ ] Test plan
[ ] Test and verify
[ ] Automation if required.

https://github.com/openbmc/openbmc-test-automation/blob/master/tests/test_bmc_file_mirroring.robot


Add your repository to the github topic among-us for better visibility.

Add your repository to the github topic among-us for better visibility.

I am not sure how to add topics to the repo. (Maybe I don't have access) @denverquane should have permission though 👍

Done, good call 👍


fix URL encoding

Use Uri.EscapeDataString even for query as GitLab there seems to handle all cases.

See
http://geekswithblogs.net/mikehuguet/archive/2009/08/16/134123.aspx
https://stackoverflow.com/questions/602642/server-urlencode-vs-httputility-urlencode

Codecov Report

Merging #105 into master will decrease coverage by 40.32%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           master     #105       +/-   ##
===========================================
- Coverage   72.61%   32.29%   -40.33%
===========================================
  Files         139      139
  Lines        1983     1728      -255
===========================================
- Hits         1440      558      -882
- Misses        543     1170      +627

Impacted Files
Coverage Δ

...t/Models/Webhooks/Requests/CreateWebhookRequest.cs
0% <ø> (ø)
:arrow_up:

...c/GitLabApiClient/Internal/Utilities/Extensions.cs
100% <100%> (ø)
:arrow_up:

...c/GitLabApiClient/Internal/Queries/QueryBuilder.cs
80% <100%> (-8.38%)
:arrow_down:

...abApiClient/Models/Projects/Responses/Namespace.cs
0% <0%> (-100%)
:arrow_down:

src/GitLabApiClient/Models/ModifiableObject.cs
0% <0%> (-100%)
:arrow_down:

...odels/MergeRequests/Requests/CreateMergeRequest.cs
0% <0%> (-100%)
:arrow_down:

...els/Groups/Requests/UpdateGroupMilestoneRequest.cs
0% <0%> (-100%)
:arrow_down:

...odels/MergeRequests/Requests/UpdateMergeRequest.cs
0% <0%> (-100%)
:arrow_down:

...tLabApiClient/Models/Releases/Responses/Release.cs
0% <0%> (-100%)
:arrow_down:

...itLabApiClient/Models/Releases/Responses/Commit.cs
0% <0%> (-100%)
:arrow_down:

... and 105 more


how to solve this when images  variable lengths

if one images has 4 , other has 6 and like this variable length how to solve this sir

@pyanishjain you can try padding the labels with a maximum length.

@pyanishjain you can try padding the labels with a maximum length.

also how to handle if the characters on the captcha may be in two rows