Intellisense does not reckognize OpenCV object members/methods

Environment data

VS Code version: 1.56.2
Extension version (available under the Extensions sidebar): v2021.5.842923320
OS and version: Win10m 64bit
Python version (& distribution if applicable, e.g. Anaconda): Python 3.9.5 (and reproduced on Python 3.8.8)
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
Relevant/affected Python packages and their versions: OpenCV
Relevant/affected Python-related VS Code extensions and their versions: Python
Value of the python.languageServer setting: Pylance

[NOTE: If you suspect that your issue is related to the Microsoft Python Language Server (python.languageServer: 'Microsoft'), please download our new language server Pylance from the VS Code marketplace to see if that fixes your issue]
Expected behaviour
Intellisense suggests members/methods (e.g. .shape or .item()) of an "img" object.
Actual behaviour
Intellisense does not suggest members/methods (e.g. .shape or .item()) of an "img" object.
Steps to reproduce:
[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

Thanks for the report. How are you installing OpenCV?
Also, could you please add the logs - Instructions
thanks

I installed OpenCV using pip using the following command from within vscode:
pip install opencv-contrib-python
I am not really sure how I can get some output in my console, but this is what I got:

Error 2021-06-09 20:09:08: stderr jediProxy Error (stderr) c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py:590: DeprecationWarning: Providing the line is now done in the functions themselves like Script(...).complete(line, column)
script = jedi.Script(
c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py:590: DeprecationWarning: Providing the column is now done in the functions themselves like Script(...).complete(line, column)
script = jedi.Script(
Error 2021-06-09 20:09:08: stderr jediProxy Error (stderr) c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py:590: DeprecationWarning: Deprecated since version 0.17.0. Use the project API instead, which means Script(project=Project(dir, sys_path=sys_path)) instead.
script = jedi.Script(
c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py:624: DeprecationWarning: Deprecated since version 0.16.0. Use Script(...).infer instead.
script.goto_definitions(), request["id"]

And here is another output shown in my console:

User belongs to experiment group 'pythonaacf'
User belongs to experiment group 'pythonJoinMailingListVar3'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'pythonSurveyNotification'
User belongs to experiment group 'PythonPyTorchProfiler'
User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'ShowExtensionSurveyPrompt - control'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'LocalZMQKernel - experiment'
User belongs to experiment group 'CollectLSRequestTiming - control'
User belongs to experiment group 'CollectNodeLSRequestTiming - experiment'
User belongs to experiment group 'EnableIPyWidgets - experiment'
User belongs to experiment group 'RunByLine - experiment'
User belongs to experiment group 'CustomEditorSupport - control'
Python interpreter path: ~\AppData\Local\Programs\Python\Python39\python.exe
Error 2021-06-09 21:25:29: Failed to check if file needs to be fixed [EntryNotFound (FileSystemError): Unable to read file 'c:\Users\Alexander\Documents\GitHub\2020_Python.vscode\settings.json' (Error: Unable to resolve non-existing file 'c:\Users\Alexander\Documents\GitHub\2020_Python.vscode\settings.json')
at _handleError (c:\Users\Alexander\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:87:156171)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async y.readText (c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\out\client\extension.js:9:344717)
at async p.doesFileNeedToBeFixed (c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\out\client\extension.js:59:581560)
at async c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\out\client\extension.js:59:580687
at async Promise.all (index 1)
at async p.getFilesToBeFixed (c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\out\client\extension.js:59:580633)
at async p.updateTestSettings (c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\out\client\extension.js:59:580260)] {
code: 'FileNotFound'
}
Starting Jedi Python language engine.
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.prefix)"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.prefix)"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py site --user-site
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py site --user-site
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\completion.py
conda --version
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py pylint --generated-members=cv2.* --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text c:\Users\Alexander\Documents\GitHub\2020_Python\Part_06_opencv\16_opencv_02.py
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
~\AppData\Local\Programs\Python\Python39\python.exe c:\Users\Alexander.vscode\extensions\ms-python.python-2021.5.842923320\pythonFiles\pyvsc-run-isolated.py pylint --generated-members=cv2.* --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text c:\Users\Alexander\Documents\GitHub\2020_Python\Part_06_opencv\16_opencv_02.py
cwd: c:\Users\Alexander\Documents\GitHub\2020_Python
##########Linting Output - pylint##########
************* Module 16_opencv_02
6,0,convention,missing-final-newline:Final newline missing
1,0,convention,invalid-name:Module name "16_opencv_02" doesn't conform to snake_case naming style
1,0,convention,missing-module-docstring:Missing module docstring

Your code has been rated at 2.50/10 (previous run: 2.50/10, +0.00)

These logs imply that you're using Jedi, not Pylance.
Are you trying to use Python 2, by chance? Have you explicitly chosen Pylance as your language server in your settings but it's not loading?

Thank you, it was indeed set to Jedi, not sure how that one got in, because I used to have Pylance. I did change it back to Pylance, and reloaded the window. I'm now using Pylance again, so far so good. Amusingly enough, though, now I get 0 suggestions.
I am not trying to use Python 2, but Python 3.9.5.

My settings.json file:
{     "C_Cpp.updateChannel": "Insiders",     "workbench.startupEditor": "newUntitledFile",     "workbench.editorAssociations": [         {             "viewType": "jupyter.notebook.ipynb",             "filenamePattern": "*.ipynb"         }     ],     "python.languageServer": "Pylance",     "python.linting.pylintArgs": ["--generated-members=cv2.*"],     "python.analysis.logLevel": "Trace" }

I was asking about Python 2 as by default Jedi will be used if that's your interepter. No worries.
Do you see that cv2. gets completions, and it's just completions on things after that that fail?

cv2.imread's return is Any so we dont know the type.

Do you see that cv2. gets completions, and it's just completions on things after that that fail?

Sorry, I was away for a couple of days, but would like to come back to this issue if possible.
cv2. gets completions as they should, everything works well there.

our generated stub for cv2.imread's return is Any so we dont know the type.

I'm sorry, but I do not quite understand. Could you elaborate please?
Thank you all.
Cheers,

try right click, go to declaration on imread. You will see the cv2 pyi type stub. the return type of the function is Any

try right click, go to declaration on imread. You will see the cv2 pyi type stub. the return type of the function is Any and therefore we don't know any member/methods of Any

Okay, I think I'm understanding a bit more of what you're saying, thank you for your patience. I'd like to add one more question, though. I tried the jupyter notebook extension in VSCode, and there the intellisense does show the member/method.
Any idea what jupyter notebook extension is doing differently compared to the regular python extension?

Jupyter actually executes the code, so has live data pulled directly from the kernel/interpreter. It then merges that info with the static analysis that Pylance does.

Jupyter actually executes the code, so has live data pulled directly from the kernel/interpreter. It then merges that info with the static analysis that Pylance does.

Okay, thank you for the explanation. Is there any possibility / option / solution to have the live data pulled directly from the kernel/interpreter just like in Jupyter or any other workaround to get the members of the OpenCV object?

For regular files, doing so would be quite dangerous; we don't want to go run your code (or libraries) automatically. With jupyter, you're in control of which code runs and does not.

For regular files, doing so would be quite dangerous; we don't want to go run your code (or libraries) automatically. With jupyter, you're in control of which code runs and does not.

Hi Jake
thank you for the answer. I am more and more willing to accept that there was a misconception from my side. One last question, just for my understanding:
You mentioned that in jupyter, I am in control of which code runs and which does not. I assume you're referring to the fact that in jupyter, one can runs cells individually. That's what you mean with control over what code runs and what not. Am I correct?
Thank you,

Yes, so you run the code because you know (or assume) it's safe to run. We don't want to auto-run code to get info from it; maybe you wrote a script or imported a library that deletes all of your files, for example.

excellent, then I think that there's nothing left to say from my side apart from a big thank you for helping me understand the nature of the issue much better.
I appreciate the responsiveness and the clarification given from your side.
Best regards,


Server gets stuck

When stopping my server, it freezes after this line:
[23:24:56] [Server thread/INFO] (ImmersiveCursedness) Stopping immersive cursedness thread
I then have to kill/terminate it for it to actually stop.
Full shutdown logs: https://paste.srnyx.xyz/ocuyaxipar.csharp

I also have this issue

Tested, the mod actually end normally, it just doesn't say it's finished or something.


Refactoring. Introduction of vscode abstraction layer & unit tests

Related to issue #17
Refactoring. Improved testability and maintainability by:

introducing a vscode abstraction layer
moved logic into different service classes (incl. dependency injection)
introducing unit tests incl code coverage (in addition to existing integration tests)

This PR likely needs to be updated once the other PRs have been merged. But this PR contains the basis for these future update and should outline the approach and recommended changes.

Thanks @The-Compiler for your comments. I agree with you that this adds an extra level of complexity. That said, there is a difference between unit tests and integration tests (using the underlaying VSCode infrastructure). Unit tests won't replace integration tests: both types are there for very specific reasons. Unit tests should be considered as an extra benefit for the developers.
Unit tests are typically the first set of tests devs run: they should be fast and external systems should be abstracted (decoupled) as much as possible. They should be fast and run often (help to increase developer velocity). if a unit test fail it doesn't make sense to run other activities that potentially are more expensive (performance and/or cost).
The focus is really to test a single unit instead of the entire flow. Unit tests make it easy to identify "potential" problematic code areas (e.g. if it is difficult to write an unit test for a specific component - thats typically a good indicator that the code base should be refactored - and overall codebase maintainability will increase over time). Let me give you another example: the VSCode integration tests make it really hard to mock certain areas (this is per-se not a problem of your code - but the way how the VSCode libraries are developed in combination with the supported testing tools/lib makes mocking harder). So, writing fully automated tests is not as easy as it could be. So, decouple your code from the VSCode integration code will definitively help here. Plus, unit tests are a great documentation tool to document the actual code base (a highly underestimated value of unit tests). You an achieve similar things with integration tests but they are not as targeted (to a single unit, function/method, etc) as unit tests.
But I get your point that this change introduces some complexity. Based on my experience I can tell: the importance of unit tests is directly correlated to (not a complete list):

to the size of the code base
number of code contributors

But again, thanks for reviewing the PR and your feedback!


BossBarCoulor not working.

BossBarColour is not working, when used is not changing the color of the job boss bar. I'm on <IP_ADDRESS>, CmiLib <IP_ADDRESS>
Minero:
  fullname: Minero
  shortname: Mi
  FullDescription:
  - "Gana dinero minando menas y minerales."
  ChatColour: DARK_GRAY
  BossBarColour: YELLOW
  chat-display: full
  max-level: 100
  leveling-progression-equation: 13*(joblevel)+(joblevel*joblevel*5)
  income-progression-equation: baseincome+(baseincome*(joblevel-2)*0.01)-((baseincome+(baseincome*(joblevel-2)*0.01))
    * ((numjobs-1)*0.01))
  points-progression-equation: basepoints+(basepoints*(joblevel-1)*0.01)-((basepoints+(basepoints*(joblevel-1)*0.01))
    * ((numjobs-1)*0.05))
  experience-progression-equation: baseexperience-(baseexperience*((numjobs-1) *0.01))
  rejoinCooldown: 10
  maxDailyQuests: 0
  Gui:
    Item: wooden_pickaxe
  Break:
    andesite:
      income: 0.1
      experience: 0.1
    granite:
      income: 0.1
      experience: 0.1
    diorite:
      income: 0.1
      experience: 0.1
    sandstone:
      income: 0.015
      experience: 0.02
    chiseled_sandstone:
      income: 0.015
      experience: 0.02
    cut_sandstone:
      income: 0.015
      experience: 0.02
    coal_ore:
      income: 0.3
      experience: 0.2
    deepslate_coal_ore:
      income: 0.4
      experience: 0.3
    redstone_ore:
      income: 0.25
      experience: 0.4
    deepslate_redstone_ore:
      income: 0.35
      experience: 0.6
    copper_ore:
      income: 0.35
      experience: 0.4
    deepslate_copper_ore:
      income: 0.45
      experience: 0.6
    iron_ore:
      income: 0.7
      experience: 0.4
    deepslate_iron_ore:
      income: 0.9
      experience: 0.6
    gold_ore:
      income: 5
      experience: 2
    deepslate_gold_ore:
      income: 6
      experience: 3
    lapis_ore:
      income: 7.5
      experience: 2
    deepslate_lapis_ore:
      income: 8.5
      experience: 3
    diamond_ore:
      income: 10
      experience: 10
    deepslate_diamond_ore:
      income: 12.5
      experience: 12.5
    emerald_ore:
      income: 15
      experience: 30
    deepslate_emerald_ore:
      income: 17.5
      experience: 35
    nether_quartz_ore:
      income: 0.5
      experience: 0.8
    nether_gold_ore:
      income: 0.5
      experience: 0.3
    ancient_debris:
      income: 200.0
      experience: 50.0

Adding to this:

This issue affects the output of /jobs stats as well
To reproduce, simply use the default configuration (delete the jobs folder and let the plugin regenerate every file)

Adding to this:

This issue affects the output of /jobs stats as well
To reproduce, simply use the default configuration (delete the jobs folder and let the plugin regenerate every file)

It's working, for the changes to apply you'll have to restart the server, reload is not working for this.

7954e62ae08837813cb5a0dfd16b105c03c26c36


Started http server

Hi Prisma Team! My Prisma Client just crashed. This is the report:
Versions

Name
Version

Node
v12.18.2

OS
rhel-openssl-1.0.x

Prisma
2.3.0

Description
Started http server

Logs
2020-08-11T05:32:48.863Z prisma-client {
  engineConfig: {
    cwd: '/var/task/src/node_modules/.prisma/client',
    enableDebugLogs: false,
    enableEngineDebugMode: undefined,
    datamodelPath: '/var/task/src/node_modules/.prisma/client/schema.prisma',
    prismaPath: undefined,
    engineEndpoint: undefined,
    generator: {
      name: 'client',
      provider: 'prisma-client-js',
      output: '/opt/build/repo/node_modules/@prisma/client',
      binaryTargets: [Array],
      previewFeatures: [Array],
      config: {}
    },
    showColors: false,
    logLevel: undefined,
    logQueries: undefined,
    flags: [],
    clientVersion: '2.3.0',
    enableExperimental: [ 'aggregations' ]
  }
}
2020-08-11T05:32:50.250Z plusX Execution permissions of /var/task/src/node_modules/.prisma/client/query-engine-rhel-openssl-1.0.x are fine
2020-08-11T05:32:50.491Z prisma-client Client Version 2.3.0
2020-08-11T05:32:50.491Z prisma-client Engine Version e11114fa1ea826f9e7b4fa1ced34e78892fe8e0e
2020-08-11T05:35:50.930Z prisma-client Error: Started http server

This is a non-recoverable error which probably happens when the Prisma Query Engine has a panic.

https://github.com/prisma/prisma-client-js/issues/new?body=Hi+Prisma+Team%21+My+Prisma+Client+just+crashed.+This+is+the+report%3A%0A%23%23+Versions%0A%0A%7C+Name+++++%7C+Version++++++++++++%7C%0A%7C----------%7C--------------------%7C%0A%7C+Node+++++%7C+v12.18.2+++++++++++%7C+%0A%7C+OS+++++++%7C+rhel-openssl-1.0.x+%7C%0A%7C+Prisma+++%7C+2.3.0++++++++++++++%7C%0A%0A%23+Description%0A%60%60%60%0AStarted+http+server%0A%60%60%60%0A%0A%23%23+Logs%0A%60%60%60%0A2020-08-11T05%3A32%3A48.863Z+prisma-client+%7B%0A++engineConfig%3A+%7B%0A++++cwd%3A+%27%2Fvar%2Ftask%2Fsrc%2Fnode_modules%2F.prisma%2Fclient%27%2C%0A++++enableDebugLogs%3A+false%2C%0A++++enableEngineDebugMode%3A+undefined%2C%0A++++datamodelPath%3A+%27%2Fvar%2Ftask%2Fsrc%2Fnode_modules%2F.prisma%2Fclient%2Fschema.prisma%27%2C%0A++++prismaPath%3A+undefined%2C%0A++++engineEndpoint%3A+undefined%2C%0A++++generator%3A+%7B%0A++++++name%3A+%27client%27%2C%0A++++++provider%3A+%27prisma-client-js%27%2C%0A++++++output%3A+%27%2Fopt%2Fbuild%2Frepo%2Fnode_modules%2F%40prisma%2Fclient%27%2C%0A++++++binaryTargets%3A+%5BArray%5D%2C%0A++++++previewFeatures%3A+%5BArray%5D%2C%0A++++++config%3A+%7B%7D%0A++++%7D%2C%0A++++showColors%3A+false%2C%0A++++logLevel%3A+undefined%2C%0A++++logQueries%3A+undefined%2C%0A++++flags%3A+%5B%5D%2C%0A++++clientVersion%3A+%272.3.0%27%2C%0A++++enableExperimental%3A+%5B+%27aggregations%27+%5D%0A++%7D%0A%7D%0A2020-08-11T05%3A32%3A50.250Z+plusX+Execution+permissions+of+%2Fvar%2Ftask%2Fsrc%2Fnode_modules%2F.prisma%2Fclient%2Fquery-engine-rhel-openssl-1.0.x+are+fine%0A2020-08-11T05%3A32%3A50.491Z+prisma-client+Client+Version+2.3.0%0A2020-08-11T05%3A32%3A50.491Z+prisma-client+Engine+Version+e11114fa1ea826f9e7b4fa1ced34e78892fe8e0e%0A%60%60%60&title=Started+http+server&template=bug_report.md

If you want the Prisma team to look into it, please open the link above 🙏

    at NodeEngine.handleRequestError (/var/task/src/node_modules/@prisma/client/runtime/index.js:1:115776)
    at async PrismaClientFetcher.request (/var/task/src/node_modules/@prisma/client/runtime/index.js:1:208913)

Hello! Can you please provide a minimal reproduction for the above crash? Unfortunately the automatically collected information in the issue does not give us much to go with here - we will need a bit more information. What were you doing? Can you share the code and schema that was active?


Added better documentation, new example, requirements need fixed adafruit_framebuf

This mostly adds a new example for drawing images when connected to a Raspberry Pi and updates requirements for the bugfixed framebuf, but also improves documentation.

Thanks


Missing error -<PHONE_NUMBER>

Error code -2147156093 is missing from the page , I was referred here by this error and it is not listed please update .

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 08b1a2dc-7fa3-20d8-d6d9-990bf1cfe34d
Version Independent ID: 98c9a0f0-35bc-eacd-9aaf-acfd4b968fee
Content: Microsoft Antimalware Extension for Windows VMs on Azure - Azure Virtual Machines
Content Source: articles/virtual-machines/extensions/iaas-antimalware-windows.md
Service: virtual-machines-windows
Sub-service: extensions
GitHub Login: @TomShinder
Microsoft Alias: tomsh

@bledMS82
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@bledMS82
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

Investigating/working on this in AB#1958451

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing. We are closing this issue for now, but feel free to comment here as necessary. #please-close


csv hangs if it gets a file handle instead of a string or if it gets a Distribution::Resource

I have tried with this:
say csv(in => "chickpea-rice.csv".IO,  sep => ';', headers => "auto", key => "Ingredient" );

But it probably does not depend on the arguments.

Updated to maybe reflect another use case.

Well, this is still here. Guess will have to fix it.


[Feature] Full system encryption using GELI

FreeBSD (including version 13.0-RELEASE) supports full system encryption out of the box using GELI.
GhostBSD should incorporate this functionality, as it's based on FreeBSD; indeed, GhostBSD-21.04.27 is based on FreeBSD 13.0-RELEASE. (It looks like most of the work needed might be in the installer.)

issue moved from ghostbsd/ghostbsd-src to ghostbsd/issues

There is easy workaround:

Maybe if you know ZFS well, I'm not sure I know what you mean. 🤔

Which part? I can explain.

Let's look more deeply about which encryption method meets end user needs best - GELI encryption or ZFS encryption.
Keep in mind when implementing GELI is it is an 'either or' option. In other words, select GELI encryption or ZFS encryption, but not both. When selecting GELI we need to grey-out the ZFS encryption option when implemented. Also, whether selecting GELI encryption or ZFS encryption, the selection will affect a backup and restore solution.
BACKGROUND
GELI Encryption:
Strengths: Full disk encryption, compatibility with all filesystems, separate key management.
Weaknesses: Performance overhead, management complexity, limited ZFS integration, challenging boot environment setup, complex key management.
ZFS Native Encryption:
Strengths: Granular dataset-level encryption, integrated with ZFS features, potentially better performance, straightforward backup with ZFS send/receive.
Weaknesses: Granularity adds complexity, limited to ZFS filesystems, performance impact, complex key management, requires compatible backup solutions.
IMPACT TO BACKUP SOLUTIONS
Here's a summary of the impact of selecting either GELI or ZFS encryption on data backup and restoration processes:
GELI Encryption:
Backup:
Decrypted State:
    Backup software can access files as if they are unencrypted when the filesystem is mounted and decrypted.
    Standard file-level backup tools can be used without modification.
Encrypted State:
    Backup the entire encrypted volume as raw data.
    This approach is less common for regular backups and more suited for disaster recovery scenarios.

Restore:
Decrypted State:
    Restore files using standard file-level restoration tools.
    Re-encryption occurs automatically as data is written back to the encrypted volume.
Encrypted State:
    Restore the entire encrypted volume.
    Requires restoring the raw encrypted data and then decrypting it to access the files.

Considerations:
Performance: Decrypting data for backups may introduce performance overhead.
Complexity: Managing full disk encryption, including boot environments, can be complex.
Compatibility: Most backup software works without issue when the filesystem is mounted and decrypted.

ZFS Native Encryption:
Backup:
Decrypted State:
    Backup software can access files as usual when datasets are mounted and decrypted.
    Standard file-level backup tools can be used.
Encrypted State:
    ZFS supports sending encrypted datasets using zfs send.
    Encrypted backups can be transferred and stored securely without needing to decrypt them first.

Restore:
Decrypted State:
    Restore files using standard file-level restoration tools.
    Re-encryption is handled by ZFS as data is written back to the encrypted datasets.
Encrypted State:
    Restore encrypted datasets using zfs receive.
    Maintains encryption properties and does not require decryption during transfer.

Considerations:
Performance: Optimized for ZFS, but encryption/decryption still adds some overhead.
Integration: Seamlessly integrates with ZFS features like snapshots, replication, and dataset management.
Granularity: Offers granular encryption at the dataset level, allowing selective encryption of data.

Summary of Impact:
GELI Encryption:
Backup: Can be done in either decrypted or encrypted state. Decrypted state is more straightforward for regular backups.
Restore: Requires re-encryption if restored from decrypted backups. Raw encrypted backups need the entire volume restored and decrypted.
Complexity: Higher due to full disk encryption, especially for boot environments and mixed filesystems.

ZFS Native Encryption:
Backup: Supports both decrypted and encrypted states. Encrypted backups using zfs send are efficient and secure.
Restore: Encrypted datasets can be restored while maintaining encryption properties. Simplifies the process with ZFS tools.
Integration: Better integration with ZFS features, offering more flexibility and ease of use.

In conclusion, if you primarily use ZFS and want to leverage its features, ZFS native encryption is likely more efficient and easier to manage for both backups and restores. If full disk encryption is required or you use a mix of filesystems, GELI may be necessary, but it adds complexity and potential performance overhead.

If I remember correctly, FreeBSD does not support to boot zfs encrypted root.

If I remember correctly, FreeBSD does not support to boot zfs encrypted root.

Exactly.
That is why - today - its GELI or nothing.
... or just encrypt /home/USER dir with ZFS - its even supported in adduser(8) script now.

Let's look more deeply (...)

Thank You for summarizing ups and downs between GELI and ZFS encryption.

This is what I've learned. The best workaround for booting FreeBSD with an encrypted ZFS root filesystem involves using an unencrypted /boot partition to store the necessary boot files. This approach ensures that the system can load the bootloader and kernel before unlocking the encrypted ZFS root. Here’s a step-by-step outline of this method:
Create Partitions:

Create a small, unencrypted UFS or ZFS partition for /boot.
Create a larger ZFS partition for the root filesystem with encryption enabled.

gpart create -s gpt ada0
gpart add -t efi -a 1m -s 256m -l efi ada0
gpart add -t freebsd-boot -a 1m -s 512k -l boot ada0
gpart add -t freebsd-ufs -a 1m -s 2G -l bootfs ada0
gpart add -t freebsd-zfs -a 1m -l zroot ada0

Format the Partitions:

Format the EFI and UFS partitions.
Create the ZFS pool with encryption.

newfs_msdos /dev/gpt/efi
newfs /dev/gpt/bootfs
zpool create -o ashift=12 -O encryption=on -O keyformat=passphrase zroot /dev/gpt/zroot

Install the Base System:

Mount the filesystems and install the base system and kernel to the ZFS pool.
Copy the necessary bootloader files to the UFS partition.

zfs create zroot/ROOT
mount -t ufs /dev/gpt/bootfs /mnt
tar -xpf /path/to/base.txz -C /mnt
tar -xpf /path/to/kernel.txz -C /mnt
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf

Configure the Bootloader:

Copy the bootloader to the EFI partition.
Configure the loader to point to the root ZFS pool.

mkdir -p /mnt/boot/efi/boot
cp /mnt/boot/loader.efi /mnt/boot/efi/boot/bootx64.efi

Finalize the Configuration:

Set the ZFS boot filesystem.
Unmount and reboot the system.

zpool set bootfs=zroot/ROOT zroot
umount /mnt
reboot

This workaround ensures that the system can boot from the unencrypted /boot partition and then unlock and mount the encrypted root filesystem.
For detailed instructions and community discussions, you can refer to the following sources:

FreeBSD Forums - Quest for Unencrypted /boot and ZFS Native Encrypted Root【7†source】.
Reddit Discussion on Booting FreeBSD with Encrypted ZFS Root【9†source】.
FreeBSD Forums - Loader.efi and Encrypted ZFS Root【10†source】.

... and this is the first step to loose one of the best FreeBSD features - ZFS Boot Environments.

Here is a suggestion for implementing GELI.
1. ghostbsd-build
No changes needed to build.sh.
2. Adding GELI to Package Lists
File: packages/mate
geli
# Other existing packages...

File: packages/xfce
geli
# Other existing packages...

File: packages/mate_oem
geli
# Other existing packages...

3. gbi (GhostBSD Installer)
File: src/lib/installer.py
import subprocess

# Existing imports and code...

def setup_geli_encryption(partition, passphrase):
    try:
        # Initialize GELI on the specified partition
        subprocess.run(['geli', 'init', '-s', '4096', partition], check=True)
        # Attach the GELI partition with the passphrase
        process = subprocess.Popen(['geli', 'attach', '-j', '-', partition], stdin=subprocess.PIPE)
        process.communicate(input=passphrase.encode())
        process.wait()
        # Format the new GELI device
        new_partition = f"{partition}.eli"
        subprocess.run(['newfs', new_partition], check=True)
    except subprocess.CalledProcessError as e:
        print(f"Error setting up GELI encryption: {e}")
        raise  # Re-raise the exception after logging it for further handling
    except Exception as e:
        print(f"Unexpected error: {e}")
        raise

# Existing installer logic...

# Add GELI encryption step if selected
if user_selected_geli_encryption:
    setup_geli_encryption(selected_partition, user_passphrase)

File: src/ui/installer_ui.py
import gi
gi.require_version('Gtk', '3.0')
from gi.repository import Gtk

# Existing imports and code...

class InstallerUI:
    def __init__(self):
        self.builder = Gtk.Builder()
        self.builder.add_from_file("installer.ui")

        # Existing UI elements...

        self.geli_encryption_checkbox = Gtk.CheckButton(label="Enable GELI Encryption")
        self.geli_encryption_checkbox.connect("toggled", self.on_geli_encryption_toggled)
        self.grid.attach(self.geli_encryption_checkbox, 0, 5, 1, 1)

        self.geli_passphrase_entry = Gtk.Entry()
        self.geli_passphrase_entry.set_visibility(False)
        self.grid.attach(self.geli_passphrase_entry, 1, 5, 1, 1)

    def on_geli_encryption_toggled(self, widget):
        if widget.get_active():
            self.geli_passphrase_entry.set_sensitive(True)
        else:
            self.geli_passphrase_entry.set_sensitive(False)

    # Existing methods...

# Main function to start the installer
def main():
    app = InstallerUI()
    Gtk.main()

if __name__ == "__main__":
    main()

4. pc-sysinstall
File: pc-sysinstall/backend/functions-disk.sh
#!/bin/sh
# pc-sysinstall disk functions

# Existing functions...

setup_geli() {
    local partition=$1
    local passphrase=$2

    # Initialize GELI
    echo "$passphrase" | geli init -s 4096 /dev/"$partition" -J -
    # Attach GELI partition
    echo "$passphrase" | geli attach /dev/"$partition" -j -

    # Format the GELI encrypted partition
    newfs /dev/"$partition".eli
}

# Example usage
if [ "$GELI_ENCRYPTION" = "YES" ]; then
    setup_geli "$PARTITION" "$GELI_PASSPHRASE"
fi

# Additional functions...

File: pc-sysinstall/conf/pc-sysinstall.conf
# pc-sysinstall configuration

# Existing configuration options...

GELI_ENCRYPTION="YES"
GELI_PASSPHRASE="user_provided_passphrase"

Since ZFS encryption is still an immature solution. Implementing GELI encryption would be our only solution.
If GELI encryption creates breakage to other applications (e.g. backup station), then that would be an issue to consider.

GELY is already implemented but doesn't work properly.


Do any of the preset operating systems support c compilers?

To familiarize myself with v86, I've been trying to write a simple c program and compile it using just the preset operating systems on the website. However, it appears I am out of my depth here as I can't seem to find any c compiler included among the presets. Is there some way to add one (e.g. to linux, freedos, etc.), or is it too difficult given the limitations of the emulator?
Thanks in advance!

The FreeBSD profile has clang and the Arch Linux profile has gcc. Does that work for you?

Yes it does, thank you! Just one other question, what are the accepted file formats for adding a hard drive disk image? Is there a recommended way to add or create one in general?

Only raw images are supported. I'd generally recommend going through the install process in qemu and using the resulting image in v86. There is some documentation for creating an Arch Linux image here: https://github.com/copy/v86/blob/master/docs/archlinux.md


Added Project wise Contribution

I have added function to fetch users progress project wise at Api -> Users -> controller.js
And add listed this route at Api -> Users -> index.js
closes #30

there is already pr opened on that issue @nishantKathrotiya


Find a fallback image for episodes without images

What are the dimensions for this image?

You also could make a grayscale image of the logo of the app as a fallback. Doesn't attract much attention and looks good, I think.

Since 2e4a69643af762c141e06fc208e728bb11cccfca, the episode thumbnails and backgrounds show static when no image can be found. It would have been awesome, but I couldn't het the animated GIF to animate.

I'm still open for better ideas (or a way to get the GIF to animate), but I'm closing this for now. Please reopen this issue if you have anything to add.


Does ATS support SSD+HDD as hot/cold cache?

Does ATS support SSD+HDD as hot/cold cache?
I read the whole documentation and it doesn't mention it, just a little configuration for SSD. However, before 6.0, SSD can be used as the middle layer. https://cwiki.apache.org/confluence/display/TS/SSDSupport

Any advice please?
While a more detailed discussion of storage stratification is covered in Cache Partitioning below, very briefly you may be able to realize gains in performance by separating more frequently accessed content onto faster disks (PCIe SSDs, for instance) while maintaining the bulk of your on-disk cache objects, which may not receive the same high volume of requests, on lower-cost mechanical drives.  ---from https://docs.trafficserver.apache.org/en/latest/admin-guide/performance/index.en.html#disk-storage
But no more...

No, ATS doesn't have a way to do tiered support for the drives.  A few companies that use ATS have setup the cache as a hierarchy with a  tier of SSD servers backed by rotational drive servers.

No, ATS doesn't have a way to do tiered support for the drives. A few companies that use ATS have setup the cache as a hierarchy with a tier of SSD servers backed by rotational drive servers.

Thanks


Added support for specifying cassandra_version="git:"

Lets try this.

I've merged this with commit 0e436d68f84a2c9c615226e6d33087f1f2bac701.
I made a few minor modifications, like using the Cassandra version for the cluster version instead of the git branch name. Thansk Tyler.


Inconsistency between throw and throw new

Describe the bug
In most cases, the Standard Library uses throw new but in some cases it directly throws the object. According to [1], it doesn't matter, but consistency in the codebase is good.
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/Error

throw_no_new.txt
Here is a quick and dirty analysis of where throw doesn't include new. Many of these are re-throws, so they don't need it.

Just checking that the search parameters are correct here. I think we should be searching for the regex throw (?!new). This should also include cases such as throw SyntaxError(), etc.

I actually did the following:
find . -type f -not -path "./coverage/*" -exec grep "throw" -H '{}' \; > ~/thrown.txt
If you just use grep, I had problems with some of the test files that include a named pipe and grep just got hungup.
I then removed all the lines with throw new:
grep -v "throw new" thrown.txt > throw_no_new.txt
From here I scanned through the list and removed .md, .mjs /testdata/. It left a few hundred that didn't take too long to filter. There were a few common patters:

One that were in comments. This was easy to spot because the line starts with a *
return this.throwError(
Text like "should not throw error"
_test.ts files

I'm pretty quick with multi-cursors in VSCode, so I just filtered those common cases by hand. It left me with things you'd expect throw Error and stuff like throw err. There were maybe 10 files, and I just went through them and checked that the throw err was really just a re-throw and not some variable that was constructed earlier. If we assume that all Errors start with a capital, then grep "throw [A-Z]" now produces:
./crypto/_wasm/lib/deno_std_wasm_crypto.generated.mjs:      throw Error("TextDecoder not available");
./crypto/_wasm/lib/deno_std_wasm_crypto.generated.mjs:      throw Error("TextEncoder not available");
./Releases.md:- fix(encoding/base64url): throw TypeError in case of invalid base64url string
./fs/move.ts:      throw EXISTS_ERROR;

The first 2 are in a generated file. 3rd one is in a README, and the last one throws a constant which is defined as:
const EXISTS_ERROR = new Deno.errors.AlreadyExists("dest already exists.");

I'm sure I missed something, but I think we can close this and if someone sees one I missed they can file an issue and I'll see if there is a pattern.

I'll close this. Thanks @iuioiua for the quick review.


Add secondary Waxman LeakSmart class without PollControl cluster for sensors that don't support it.

This adds a secondary LeakSmart class in the quirk to support sensor models without the PollControl cluster.

Tested to work with my LeakSmart sensors.

Thanks!


Graceful shutdown for all the services

Currently, we do nothing when users try to shut down services such as PMS, ADS. We need to handle a signal SIGQUIT and shut down services gracefully.

The pull request was merged, close this issue.


(CSS) missing properties

I noticed some CSS properties not being highlighted properly. I found css-shared.js to be missing some properties. Mainly it concerns CSS Grid properties.
Example of produced highlighting:

Expected behavior:

As CSS is nowadays defined by numerous standards and drafts, it’s debatable what should be added. Currently I’m preparing a pull request for this, and I want to discuss what to add.
My solution
To keep the PR simple, I will only add missing CSS properties to css-shared.js. I’m using https://www.w3.org/Style/CSS/all-properties.en.html as my source, which is more useful than the reference of MDN. The latter isn’t complete and contains more than only properties.
Only properties that are part of a W3C Recommendation or Candidate Recommendation will be added. This choice can be seen as conservative, as some properties already listed (like overflow-x) are part of a specification that’s still an Editor’s Draft.
Because the aural part of CSS 2.1 has been deprecated, I will skip those properties and use the properties from CSS Speech Module instead. Other specifications that I’ll consult are:

CSS Basic User Interface Module Level 3 (CSS3 UI)
CSS Cascading and Inheritance Level 3
CSS Fonts Module Level 3
CSS Grid Layout Module Level 1
CSS Masking Module Level 1
CSS Scroll Snap Module Level 1
CSS Shapes Module Level 1
CSS Template Layout Module
CSS Text Module Level 3
CSS Transforms Module Level 1
CSS Will Change Module Level 1
CSS Writing Modes Level 3
CSS Writing Modes Level 4
Compositing and Blending Level 1

I will be adding around a hundred properties. Any thoughts on the above?

Only properties that are part of a W3C Recommendation or Candidate Recommendation will be added. This choice can be seen as conservative, as some properties already listed (like overflow-x) are part of a specification that’s still an Editor’s Draft.

I'd say if multiple mainstream browser engines support it then it's "ok"... but if we're "adding in bulk" as we are here I'm fine with being more conservative.  The idea is for our grammars to be useful for 99% of people, not necessarily exhaustive because often the size difference between those two is quite large.
Please submit your PR and I'll have a look.

Probably but I think the sticky/tricky part is deciding which are immediately useful without just adding everything in the universe (and bloating the grammar for things no one needs/uses)... so I don't know that we have an exact target as to what needs to be done here...  it's a bit nebulous.


[ZEPPELIN-5875] Fix flaky tests in core-modules

What is this PR for?
Fixing flaky test in core-modules (hadoop[2|3]).
What type of PR is it?
Bug Fix
Todos

[x] - Fix testTimeout_2

What is the Jira issue?

ZEPPELIN-5875

How should this be tested?

All CI should pass

Screenshots (if appropriate)

Questions:

Does the license files need to update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

CI is all green now.

Thanks @jongyoul

Since the shading still creates weird classes, I will continue to work on #4545 .

Agreed. It would be great if we could handle all thing this time.


User can disable notifications for specific sender

It is possible to disable the notifications for Softwarecenter or Powershell using the sender notification settings or "Turn off all notifications setting" of the Notification Center.  You can see the registry keys beeing reset after the next run of the Toast-Notification-Script, but this does not affect the visibility of the notifications. No reboot or service restart can fix this issue. The only way to turn the notifications back on is by using the On/Off Switch in the sender notification settings.

Yeah, I know that. The built-in functionality I offer only prevents that one option from being disabled. There are other means though in completely disabling the user's options to disable the notification. I mention that in the end of this post: https://www.imab.dk/windows-10-toast-notification-script-update-second-action-button-and-built-in-prevention-from-disabling-toast-notifications/

Yeah, I know that. The built-in functionality I offer only prevents that one option from being disabled. There are other means though in completely disabling the user's options to disable the notification. I mention that in the end of this post: https://www.imab.dk/windows-10-toast-notification-script-update-second-action-button-and-built-in-prevention-from-disabling-toast-notifications/


Split CC/CXX environment variable by space characters

People sometimes set CC="ccache gcc" or CC="distcc gcc", but gcc-rs currently doesn't recognize arguments in CC/CXX and treats the whole value as the compiler executable.
With this patch, CC/CXX environment variables will be splitted by whitespace charactors. The first string is used as the compiler, and the rest are used as arguments for the compiler.

I just noticed the current README explicitly says that no extra flags can be passed inside CC.
If it is intentional behavior, please reject this, sorry.

Ah yeah currently with behavior like this we unfortunately wouldn't have the ability to specify paths with spaces in them. For now we'll stick with the current behavior, but I'd be fine adding ccache-specific options if this is common enough!

I understand, thanks for the explaination!
I have a Makefile which calls cargo build inside it, and encountered this when I ran make CC="ccache gcc" as usual I do for other C projects.


Refactoring for make-hand-service.py

Makes the following changes:

Logging with print statements.
Use tempfiles where appropriate.
Add cli options.

I'm currently running the script with:
python make_hand_service.py /home/arcgis/jtherrmann/ --rasters-filter='.*Copernicus_DSM_COG_10_N08_00_W01*'

But it fails; full log output:
CreateFileGDB
CreateMosaicDataset
AddFields
AddRastersToMosaicDataset
CalculateFields
BuildFootprints
Traceback (most recent call last):
  File "make_hand_service.py", line 90, in <module>
    update_boundary='UPDATE_BOUNDARY',
  File "/home/arcgis/miniconda3/envs/arcpy/lib/python3.7/site-packages/arcpy/management.py", line 14697, in BuildFootprints
    raise e
  File "/home/arcgis/miniconda3/envs/arcpy/lib/python3.7/site-packages/arcpy/management.py", line 14694, in BuildFootprints
    retval = convertArcObjectToPythonObject(gp.BuildFootprints_management(*gp_fixargs((in_mosaic_dataset, where_clause, reset_footprint, min_data_value, max_data_value, approx_num_vertices, shrink_distance, maintain_edges, skip_derived_images, update_boundary, request_size, min_region_size, simplification_method, edge_tolerance, max_sliver_size, min_thinness_ratio), True)))
  File "/home/arcgis/miniconda3/envs/arcpy/lib/python3.7/site-packages/arcpy/geoprocessing/_base.py", line 512, in <lambda>
    return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: ERROR 999999: Something unexpected caused the tool to fail. Contact Esri Technical Support (http://esriurl.com/support) to Report a Bug, and refer to the error help for potential solutions or workarounds.
No mosaic dataset item selected. This operation cannot be performed.
Failed to execute (BuildFootprints).

Start Time: Wednesday, February 22, 2023 9:16:36 PM
ERROR 999999: Something unexpected caused the tool to fail. Contact Esri Technical Support (http://esriurl.com/support) to Report a Bug, and refer to the error help for potential solutions or workarounds.
No mosaic dataset item selected. This operation cannot be performed.
Failed to execute (BuildFootprints).
Failed at Wednesday, February 22, 2023 9:16:38 PM (Elapsed Time: 2.59 seconds)

I wonder if perhaps the regex filter is not matching any rasters? This will be my next line of investigation.

Drop the .s from the filter, I suspect. Per https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/add-rasters-to-mosaic-dataset.htm by default you're doing simple wildcard matching, e.g. '*Copernicus_DSM_COG_10_N08_00_W01*'. It's only if you put the word REGEX` in the filter that you get full perl regular expressions.

Drop the .s from the filter, I suspect. Per https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/add-rasters-to-mosaic-dataset.htm by default you're doing simple wildcard matching, e.g. '*Copernicus_DSM_COG_10_N08_00_W01*. It's only if you put the word REGEX in the filter that you get full perl regular expressions.

Ah, I see, I added REGEX because I thought you just left that part out of the filter you gave me. So I thought you had just forgotten the ..

I ran it as python make_hand_service.py --dataset-name GLO30_HAND_ASJ --rasters-filter '*Copernicus_DSM_COG_10_N08_00_W01*' /home/arcgis/asjohnston/gis-services/image_services/glo_30_hand/ and everything worked how I expected. The .sd file published with no issues.


Should we isolate system messages?

All messages with a type of non-zero should not be processed as if they were user messages. Is this enough to a warrant a new function like DiscordSocketClient.SystemMessageReceived, or should we keep it unified in one place?

At the moment, usage is limited, but I think it's clear that it won't stay that way. I'm for moving isolating

I think separating the events would make sense. After all, most developers who are looking to create a bot won't be interested in the other message types.

Ehhh, that's debateable. But SystemMessages aren't what people are expecting in MessageReceived, I think

And the 1-2% of devs that do want to hook into system messages should be knowledgeable enough to find the new event by themselves.


Render test setup

Addressed comments by splitting RenderTest into ImageComparison and ImageLoader under testing/. Can later add more infrastructure to generate and display diffs, approve changes, etc.
Still probably doesn't work on Windows without a runfiles fix or workaround.

New tests pass on Windows now


[Feature Request]: Add 'metadata' to the available parameters.

I'm trying to work with the Slack Open ID and have run into an issue; they're not serving the proper CORS headers on their discovery endpoint https://slack.com/.well-known/openid-configuration
The project that this library is a wrapper for makes an alternative available with the metadata parameter but it doesn't seem to be supported yet. I added it and am making a pull request, contribution notes mentioned opening a ticket first!
One note, you'll still need to send an authority value over. This configuration for Slack is now working for me after the change.
  onSignIn: async (user) => {
    alert('You just signed in, congratz! Check out the console!');
    console.log(user);
    window.location.hash = '';
  },
  authority: 'https://slack.com',
  metadata: {
    issuer: 'https://slack.com',
    authorization_endpoint: "https://slack.com/openid/connect/authorize",
    token_endpoint: "https://slack.com/api/openid.connect.token",
    userinfo_endpoint: "https://slack.com/api/openid.connect.userInfo",
    jwks_uri: "https://slack.com/openid/connect/keys",
  },
  clientId: process.env.REACT_APP_SLACK_CLIENT_ID,
  responseType: 'code',
  redirectUri:
    process.env.NODE_ENV === 'development'
      ? 'https://localhost:3000/slack/authenticate'
      : 'https://localhost:3000/slack/authenticate',
};

Pull request opened here https://github.com/bjerkio/oidc-react/pull/715


fix(charts): screenshot API declaring an unnecessary rison arg

SUMMARY
Fix chart screenshot API endpoint, issue has more detail
ADDITIONAL INFORMATION

[ ] Has associated issue: #11929
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #11931 (13dcf98) into master (77d362d) will decrease coverage by 13.27%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master   #11931       +/-   ##
===========================================
- Coverage   67.64%   54.36%   -13.28%
===========================================
  Files         932      421      -511
  Lines       45211    14812    -30399
  Branches     4333     3822      -511
===========================================
- Hits        30583     8053    -22530
+ Misses      14525     6759     -7766
+ Partials      103        0      -103

Flag
Coverage Δ

cypress
54.36% <ø> (-0.51%)
:arrow_down:

javascript
?

python
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...uperset-frontend/src/dashboard/util/dnd-reorder.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...rset-frontend/src/dashboard/util/getEmptyLayout.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...dashboard/components/resizable/ResizableHandle.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

.../src/dashboard/util/getFilterScopeFromNodesTree.js
0.00% <0.00%> (-93.48%)
:arrow_down:

...src/dashboard/components/gridComponents/Header.jsx
10.52% <0.00%> (-86.85%)
:arrow_down:

...rc/dashboard/components/gridComponents/Divider.jsx
13.33% <0.00%> (-86.67%)
:arrow_down:

...ontend/src/dashboard/util/getDashboardFilterKey.ts
14.28% <0.00%> (-85.72%)
:arrow_down:

...nd/src/views/CRUD/data/query/QueryPreviewModal.tsx
14.70% <0.00%> (-82.97%)
:arrow_down:

...set-frontend/src/views/CRUD/welcome/EmptyState.tsx
5.71% <0.00%> (-82.10%)
:arrow_down:

...et-frontend/src/SqlLab/components/TableElement.jsx
4.70% <0.00%> (-81.35%)
:arrow_down:

... and 778 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77d362d...13dcf98. Read the comment docs.


Spherical coordinates fatal error

Good day!
I am trying to run a blast simulation in spherical simulation with mpi, and I get a fatal error initializing Mesh trying to run the following input mesh:

nx1        = 640         # Number of zones in X1-direction
x1min      = 0.0        # minimum value of X1
x1max      = 100.0        # maximum value of X1
ix1_bc     = outflow    # inner-X1 boundary flag
ox1_bc     = outflow    # outer-X1 boundary flag
nx2        = 640             # Number of zones in X2-direction
x2min      = 0  # minimum value of X2
x2max      = 1.5708  # maximum value of X2
ix2_bc     = polar        # inner-X2 boundary flag
ox2_bc     = outflow        # outer-X2 boundary flag
nx3        = 640             # Number of zones in X3-direction
x3min      = 0 # minimum value of X3
x3max      = 6.283185307179586  # maximum value of X3
ix3_bc     = periodic        # inner-X3 boundary flag
ox3_bc     = periodic         # outer-X3 boundary flag

nx1 = 320
nx2 = 320
nx3 = 320
I believe I have the boundary conditions right, but I still get a fatal error.
I get the same error trying to run a full sphere
x2min      = 0  # minimum value of X2
x2max      = 3.141592653589793 # maximum value of X2
ix2_bc     = polar        # inner-X2 boundary flag
ox2_bc     = polar       # outer-X2 boundary flag
Your help help with be appreciated.
Regards,
Yotam.

Spherical coordinates in Athena++ do not support going all the way to r = 0. If you want to simulate a sink at the origin, you need to choose a nonzero cutoff radius. In general, though, spherical coordinates may be a poor choice for a problem that goes close to the origin, since the timestep will be very small there.
If the problem persists after increasing x1min, please include the exact steps to replicate it, including the configure options used and the input file, as well as the error message.

Thanks for the quick reply, I tried again without the origin, there were no issues.
I wish to create a simulation around the origin, with a gravitational field along the R direction. Naturally I went to the spherical coordinates first. How do you recommend proceeding ? Can it be only done in cartesian coordinates? constructing a gravitational pull from the origin would be more challenging without spherical coordinates, even your own point source is only available on spherical coordinates.
Thanks again for the support.


Update tutorial-v2-angular-auth-code.md

Updated the possible values of initialNavigation in the routing module, since the value "enabled" is not supported anymore. Instead using either "enabledBlocking" or "enabledNonBlocking".

@HamzaBoukraa-CR : Thanks for your contribution! The author(s) have been notified to review your proposed change.

IMPORTANT: When this content is ready to merge, the reviewer must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Update howto-configure-publisher-domain.md

Publisher Domain Doc Improvement

@obrianokoyo : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@rwike77
Can you review the proposed changes? Note that the HTML-formatted links must be converted to Markdown before merging.
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
#assign:@MicrosoftDocs/public-repo-pr-review-team

The following users don't have valid GitHub IDs, or are not collaborators on this repo: microsoftdocs/public-repo-pr-review-team

@obrianokoyo I emailed you internally a little while ago.  Looks like there’s some conflicts now.  Would you still like to make the updates?  Would you mind opening up a new PR in the private repo and closing this PR?  Thanks

#assign-reviewer:@Dickson-Mwendia


Reset button state on exit tree

Steps to reproduce a bug:

Hold / hover the button
Remove it (or parent node) from the tree
When we return the button back to the tree, it's pressed / hover

Looks good, but please squash commits together. See PR workflow.

Thanks!


Process image manifest if platform not specified

If --platform is not given on 'check container', preflight will now process an manifest list if that's what the URL points to. If platform is specified, only process that platform. It will not do anything different yet for submission. It would be the equivalent of running preflight on each of the images in the manifest, with the 'arch' specified.
This fixes the reported issue. However, more will be implemented when Pyxis and all other systems are ready to assosciate the images with an actual manifest list.
Fixes #955

/retest

The error in the DCI job occurred because the JUnit file was not generated by preflight. It seems like preflight did not generate the JUnit file because it did not catch the errors generated by HasNoProhibitedPackages:

time=\"2023-07-05T15:13:27Z\" level=info msg=\"check completed\" check=HasNoProhibitedPackages err=\"unable to get a list of all packages in the image: could not get rpm list: stat /tmp/preflight-779142393/fs/var/lib/rpm/Packages: no such file or directory\" result=ERROR

@bcrochet Can we add Architecture details in log so it will be easy to identify which arch image failed or have issues?
Example run: http://pastebin.test.redhat.com/1105130

FAILURE https://www.distributed-ci.io/jobs/9385c353-c394-443e-affc-9b1c1bab2859/jobStates
from Process image manifest if platform not specified #997

This failure is because the DCI agent is expecting the results-junit.xml in the /assets/ directory, but this change is now placing them in /assets/<ARCH>/ when --platform is not used.
I could add that in the agent, but I was wondering if would it make sense to infer the arch from the system running the check. The reasoning behind that is that most of the container clients do that when pulling, running, copying a manifest list, unless explicitly requested to do all archs.
Examples:

https://docs.podman.io/en/latest/markdown/podman-pull.1.html#arch-arch
https://www.mankier.com/1/skopeo-copy#--all

Thoughts?

FAILURE https://www.distributed-ci.io/jobs/9385c353-c394-443e-affc-9b1c1bab2859/jobStates
from Process image manifest if platform not specified #997

This failure is because the DCI agent is expecting the results-junit.xml in the /assets/ directory, but this change is now placing them in /assets/<ARCH>/ when --platform is not used.

I could add that in the agent, but I was wondering if would it make sense to infer the arch from the system running the check. The reasoning behind that is that most of the container clients do that when pulling, running, copying a manifest list, unless explicitly requested to do all archs.
Examples:
* https://docs.podman.io/en/latest/markdown/podman-pull.1.html#arch-arch

* https://www.mankier.com/1/skopeo-copy#--all

Thoughts?

We already infer the arch that way. In fact, I actually would expect it to always write to '/assets/'. I'll check why it isn't. So if it isn't, I do consider that a bug. I'll take a look tomorrow.

@bcrochet Can we add Architecture details in log so it will be easy to identify which arch image failed or have issues? Example run: http://pastebin.test.redhat.com/1105130

Should be done.
As this is a public repo, please refrain from pasting internal-only URLs.

Hi @bcrochet I'm having some issues understanding the new flag and would like some clarification, the help shows:
  --platform string      Architecture of image to pull. Defaults to runtime platform. (default "amd64")

I see platform is expecting a string, so I'm not sure when the default value shown in the help takes place, could you help me understand this?
My understanding of this change is that when an image using manifest list is used in the container check if not --platform <ARCH> is passed then all the architectures offered by the manifest list will be checked, am I getting this right?
Initially, I was thinking that it will default to the runtime arch if a manifest list image was found and only switch to all the archs if passing a flag to maintain the original behavior, IMO, this would be nice. Would be this feasible? :wink:

Side notes
For a test I did, if --platform is specified on an non-manifest-list image only writes the output in the platform specified regardless of the true arch the image is, example:
## This is an amd64 image
preflight \
  --artifacts /tmp/ \
  check container \
  --loglevel trace \
  --platform none \
  quay.io/opdev/simple-demo-operator@sha256:072aedb6fdeb14fedd859ed9ff6704a5ae3d3453463d9b618308c4a261d997b4

-- snip --
time="2023-07-27T16:00:12Z" level=info msg="running checks for  for platform none"
-- snip --
time="2023-07-27T16:00:16Z" level=trace msg="image config written to disk" filename=/tmp/none/cert-image.json
time="2023-07-27T16:00:16Z" level=trace msg="rpm manifest written to disk" filename=/tmp/none/rpm-manifest.json
...

Another test this time using a ppc64le and not setting the platform (aha! moment, I guess this is where the default is taking place)
## This is a ppc64le  image
preflight \
  --artifacts /tmp/ \
  check container \
  --loglevel trace \
  quay.io/opdev/simple-demo-operator@sha256:953bba5298c53d7ed578e81131078ab697a5b34558e559237687a389e9ff7cee

-- snip --
time="2023-07-27T16:07:57Z" level=info msg="running checks for  for platform amd64"
-- snip --
time="2023-07-27T16:08:02Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-27T16:08:02Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
...

Lastly, would it be possible to have this flag as an ENV as some other flags (e.g. PFLT_ARTIFACTS), the reason being is that this way in the DCI app agent that consumes preflight we could pass an ENV and maintain this change and a release that still does not have this flag.
Sorry for the lengthy comment, hope this helps. Thanks!

FAILURE https://www.distributed-ci.io/jobs/9385c353-c394-443e-affc-9b1c1bab2859/jobStates
from Process image manifest if platform not specified #997

This failure is because the DCI agent is expecting the results-junit.xml in the /assets/ directory, but this change is now placing them in /assets/<ARCH>/ when --platform is not used.
I could add that in the agent, but I was wondering if would it make sense to infer the arch from the system running the check. The reasoning behind that is that most of the container clients do that when pulling, running, copying a manifest list, unless explicitly requested to do all archs.
Examples:
* https://docs.podman.io/en/latest/markdown/podman-pull.1.html#arch-arch

* https://www.mankier.com/1/skopeo-copy#--all

Thoughts?

I believe this to be addressed in the current version of this patch. Would you mind verifying that it is working to your expectations? It should now always put assets by default to <assets_dir>/

Hi @bcrochet I'm having some issues understanding the new flag and would like some clarification, the help shows:
  --platform string      Architecture of image to pull. Defaults to runtime platform. (default "amd64")

I see platform is expecting a string, so I'm not sure when the default value shown in the help takes place, could you help me understand this?

So, that help text should change. I'll make that happen.

My understanding of this change is that when an image using manifest list is used in the container check if not --platform <ARCH> is passed then all the architectures offered by the manifest list will be checked, am I getting this right?

That is correct. That would be the expected behavior.

Initially, I was thinking that it will default to the runtime arch if a manifest list image was found and only switch to all the archs if passing a flag to maintain the original behavior, IMO, this would be nice. Would be this feasible? 😉

Internally the dev team talked about this and we decided that having the default behavior be to process a manifest list if --platform was not specified was the way to go. We already have that flag, and didn't really want to add yet another flag. By doing so, we would be introducing yet a third way of processing. That just complicates more than we wished.

Side notes
For a test I did, if --platform is specified on an non-manifest-list image only writes the output in the platform specified regardless of the true arch the image is, example:
## This is an amd64 image
preflight \
  --artifacts /tmp/ \
  check container \
  --loglevel trace \
  --platform none \
  quay.io/opdev/simple-demo-operator@sha256:072aedb6fdeb14fedd859ed9ff6704a5ae3d3453463d9b618308c4a261d997b4

-- snip --
time="2023-07-27T16:00:12Z" level=info msg="running checks for  for platform none"
-- snip --
time="2023-07-27T16:00:16Z" level=trace msg="image config written to disk" filename=/tmp/none/cert-image.json
time="2023-07-27T16:00:16Z" level=trace msg="rpm manifest written to disk" filename=/tmp/none/rpm-manifest.json
...

Another test this time using a ppc64le and not setting the platform (aha! moment, I guess this is where the default is taking place)
## This is a ppc64le  image
preflight \
  --artifacts /tmp/ \
  check container \
  --loglevel trace \
  quay.io/opdev/simple-demo-operator@sha256:953bba5298c53d7ed578e81131078ab697a5b34558e559237687a389e9ff7cee

-- snip --
time="2023-07-27T16:07:57Z" level=info msg="running checks for  for platform amd64"
-- snip --
time="2023-07-27T16:08:02Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-27T16:08:02Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
...

That definitely sounds like a bug! Thank you for finding that.

Lastly, would it be possible to have this flag as an ENV as some other flags (e.g. PFLT_ARTIFACTS), the reason being is that this way in the DCI app agent that consumes preflight we could pass an ENV and maintain this change and a release that still does not have this flag.
Sorry for the lengthy comment, hope this helps. Thanks!

While not documented, all of the flags should automatically have a PFLT_ equivalent. Have you tried with PFLT_PLATFORM? I'll give that a shot too to verify.

While not documented, all of the flags should automatically have a PFLT_ equivalent. Have you tried with PFLT_PLATFORM? I'll give that a shot too to verify.

I did, but didn't work as I expected, does not seem like is using it:
$ preflight --version
preflight version 79d63f0 <commit: 79d63f024ae7e76a81a22dde59861b533aac70f6>

## as an env
$ env PFLT_PLATFORM=amd64 preflight   --artifacts /tmp/   check container   --loglevel trace   quay.io/opdev/simple-demo-operator:latest 2>&1 | grep -P 'platform|filename'
time="2023-07-28T17:06:03Z" level=debug msg="manifest list detected, checking all platforms in manifest"
time="2023-07-28T17:06:03Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform amd64"
time="2023-07-28T17:06:08Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-28T17:06:08Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
time="2023-07-28T17:06:09Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache
time="2023-07-28T17:06:11Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform ppc64le"
time="2023-07-28T17:06:16Z" level=trace msg="image config written to disk" filename=/tmp/ppc64le/cert-image.json
time="2023-07-28T17:06:16Z" level=trace msg="rpm manifest written to disk" filename=/tmp/ppc64le/rpm-manifest.json
time="2023-07-28T17:06:17Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache

## as a flag
$ preflight --platform amd64   --artifacts /tmp/   check container   --loglevel trace   quay.io/opdev/simple-demo-operator:latest 2>&1 | grep -P 'platform|filename'
time="2023-07-28T17:06:34Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform amd64"
time="2023-07-28T17:06:38Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-28T17:06:38Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
time="2023-07-28T17:06:39Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache

While not documented, all of the flags should automatically have a PFLT_ equivalent. Have you tried with PFLT_PLATFORM? I'll give that a shot too to verify.

I did, but didn't work as I expected, does not seem like is using it:
$ preflight --version
preflight version 79d63f0 <commit: 79d63f024ae7e76a81a22dde59861b533aac70f6>

## as an env
$ env PFLT_PLATFORM=amd64 preflight   --artifacts /tmp/   check container   --loglevel trace   quay.io/opdev/simple-demo-operator:latest 2>&1 | grep -P 'platform|filename'
time="2023-07-28T17:06:03Z" level=debug msg="manifest list detected, checking all platforms in manifest"
time="2023-07-28T17:06:03Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform amd64"
time="2023-07-28T17:06:08Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-28T17:06:08Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
time="2023-07-28T17:06:09Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache
time="2023-07-28T17:06:11Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform ppc64le"
time="2023-07-28T17:06:16Z" level=trace msg="image config written to disk" filename=/tmp/ppc64le/cert-image.json
time="2023-07-28T17:06:16Z" level=trace msg="rpm manifest written to disk" filename=/tmp/ppc64le/rpm-manifest.json
time="2023-07-28T17:06:17Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache

## as a flag
$ preflight --platform amd64   --artifacts /tmp/   check container   --loglevel trace   quay.io/opdev/simple-demo-operator:latest 2>&1 | grep -P 'platform|filename'
time="2023-07-28T17:06:34Z" level=info msg="running checks for quay.io/opdev/simple-demo-operator:latest for platform amd64"
time="2023-07-28T17:06:38Z" level=trace msg="image config written to disk" filename=/tmp/amd64/cert-image.json
time="2023-07-28T17:06:38Z" level=trace msg="rpm manifest written to disk" filename=/tmp/amd64/rpm-manifest.json
time="2023-07-28T17:06:39Z" level=trace msg="prefix and suffix excluded" filename=usr/lib64/gconv/gconv-modules.cache prefix=usr/ suffix=.cache

It does work. Technically. However, I have some updates to this patch (not yet pushed) that enable it to work.

While not documented, all of the flags should automatically have a PFLT_ equivalent. Have you tried with PFLT_PLATFORM? I'll give that a shot too to verify.

I did, but didn't work as I expected, does not seem like is using it:

Please try again?
And now, if you specify --platform on an image, that's not a manifest list, it will error. We can't validate that the arch is correct, as that is not something in the manifest.

One more issue, though I'm unsure if this is a related bug off-hand. Preflight will still pull down an image that isn't built for the runtime architecture without complaint, even if the --platform flag isn't set.
E.g.
quay.io/komish/simple-demo-operator-ppc:0.0.1

This contains only the PPC image from simple-demo-operator. Preflight pulls it down, and states its doing so for amd64.
time="2023-08-01T13:02:42-05:00" level=info msg="certification library version" version="0.0.0 <commit: fc01e4634438397209bbdbf90d72ba96ad8f2d41>"
time="2023-08-01T13:02:42-05:00" level=info msg="running checks for quay.io/komish/simple-demo-operator-ppc:0.0.1 for platform amd64"
time="2023-08-01T13:02:42-05:00" level=info msg="target image" image="quay.io/komish/simple-demo-operator-ppc:0.0.1"

On one hand, I think I'd expect it to throw an error for a non-index image that isn't built for the same platform/architecture. On the other hand, podman pulls the image and just states the following:
WARNING: image platform (linux/ppc64le) does not match the expected platform (linux/amd64)

If we decide to just pull it anyway, then I think I just expect the log line to be correct in stating the pulled architecture.

And now, if you specify --platform on an image, that's not a manifest list, it will error. We can't validate that the arch is correct, as that is not something in the manifest.

I've validated this, and is working as you describe, thanks!

There's an additional thing that I found. the PFLT_DOCKERCONFIG seems to have been modified:
This is an stable image of preflight:
$ podman run   --rm  -e PFLT_LOGLEVEL=TRACE  -e PFLT_DOCKERCONFIG=/opt/config.json -v ${PWD}/preflight_tmp_dir.72lh398q/config.json:/opt/config.json   quay.io/opdev/preflight:stable   check container   docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f 2>&1 |head -5
time="2023-08-01T23:36:50Z" level=debug msg="config file not found, proceeding without it"
time="2023-08-01T23:36:50Z" level=info msg="certification library version" version="1.6.11 <commit: 9234b2e92055178758fed5f298f5077d01f55a60>"
time="2023-08-01T23:36:50Z" level=info msg="target image" image="docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f"
time="2023-08-01T23:36:50Z" level=debug msg="pulling image from target registry"
time="2023-08-01T23:36:50Z" level=trace msg="entering preflight keychain Resolve"

This is with an image of this commit:
$ podman run   --rm  -e PFLT_LOGLEVEL=TRACE  -e PFLT_DOCKERCONFIG=/opt/config.json -v ${PWD}/preflight_tmp_dir.72lh398q/config.json:/opt/config.json   localhost/preflight-manifest-list:latest   check container   docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f 2>&1 |head -5
time="2023-08-01T23:35:42Z" level=debug msg="config file not found, proceeding without it"
time="2023-08-01T23:35:42Z" level=info msg="certification library version" version="198c161 <commit: 198c161eebea1f4e1c19422b0e69452b671852da>"
Error: invalid manifest?: GET https://index.docker.io/v2/ibmcom/ibm-mq-operator/manifests/sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Usage:
  preflight check container [flags]

In both cases I'm using auth creds for docker to avoid hitting the limit, in the test for this image it does not seem to be using them, any idea?

...
Error: invalid manifest?: GET https://index.docker.io/v2/ibmcom/ibm-mq-operator/manifests/sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
...
In both cases I'm using auth creds for docker to avoid hitting the limit, in the test for this image it does not seem to be using them, any idea?

@tonyskapunk can you confirm that the authentication you're using is not hitting rate limits? https://docs.docker.com/docker-hub/download-rate-limit/#:~:text=Docker Hub limits the number,pulls per 6 hour period.
@bcrochet Is it possible that the early crane calls to resolve the manifest and make decisions on how to handle an index are unauthenticated causing this problem?

@tonyskapunk can you confirm that the authentication you're using is not hitting rate limits? https://docs.docker.com/docker-hub/download-rate-limit/#:~:text=Docker Hub limits the number,pulls per 6 hour period.

@komish, I confirm the auth we have avoids the rate limit.  I also tested with skopeo where I see the limit error when not using the auth file.
$ skopeo inspect --authfile config.json docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f | jq .Name
"docker.io/ibmcom/ibm-mq-operator"

$ skopeo inspect docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f

FATA[0031] Error parsing image name "docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f": reading manifest sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f in docker.io/ibmcom/ibm-mq-operator: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

hth

@komish Yeah, they are unauthenticated calls since we aren't passing any options to the methods, looks like we need to build crane.Options, with at least the dockerconfig earlier in the flow. I would hesitate to build it identical to what we have in the CraneEngine.
I think the change needs to happen here

Hi @bcrochet , I'd like to understand a bit more the change.

preflight will now process an manifest list if that's what the URL points to.
It will not do anything different yet for submission.

If I understand correctly, this change will generate several folders, one per architecture, and each folder will contain results.json, something like this
server
├── amd64
│      └── results.json
├── arm64
│      └── results.json

Which of these results.json files will be submitted to the Connect UI with Pyxis?

@tkrishtop also, to clarify, I believe the original PR message by @bcrochet is no longer accurate. I believe this PR does submit each asset as @acornett21 describes.

@tonyskapunk can you check the latest push to make sure it gets your past your rate limiting issue?

@tkrishtop also, to clarify, I believe the original PR message by @bcrochet is no longer accurate. I believe this PR does submit each asset as @acornett21 describes.

I have updated the PR description to cover this.

@tonyskapunk can you check the latest push to make sure it gets your past your rate limiting issue?

Before testing I made sure I still see the limit issue:
$ skopeo inspect docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f
FATA[0031] Error parsing image name "docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f": reading manifest sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f in docker.io/ibmcom/ibm-mq-operator: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

$ skopeo inspect --authfile config.json docker://docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f | jq .Name
"docker.io/ibmcom/ibm-mq-operator"

And now preflight is happy :)
$ podman run --privileged --rm -e PFLT_LOGLEVEL=TRACE  -e PFLT_DOCKERCONFIG=/opt/config.json -v ${PWD}/config.json:/opt/config.json localhost/preflight-manifest-list:latest check container   docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f 2>&1 | head -10
time="2023-08-03T17:58:05Z" level=debug msg="config file not found, proceeding without it"
time="2023-08-03T17:58:05Z" level=info msg="certification library version" version="4f72cda <commit: 4f72cdaf6d20a1f521b687715126edcabdf4e10b>"
time="2023-08-03T17:58:05Z" level=trace msg="entering preflight keychain Resolve"
time="2023-08-03T17:58:05Z" level=debug msg="manifest list detected, checking all platforms in manifest"
time="2023-08-03T17:58:05Z" level=info msg="running checks for docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f for platform s390x"
time="2023-08-03T17:58:05Z" level=info msg="target image" image="docker.io/ibmcom/ibm-mq-operator@sha256:bc236a217cd8a46a715efff5cebaa20399e6e91cbf0345de05943b17774dd83f"
time="2023-08-03T17:58:05Z" level=debug msg="pulling image from target registry"
time="2023-08-03T17:58:05Z" level=trace msg="entering preflight keychain Resolve"
time="2023-08-03T17:58:05Z" level=debug msg="created temporary directory" path=/tmp/preflight-1174448130
time="2023-08-03T17:58:05Z" level=debug msg="exporting and flattening image"

You can also see the DCI  job here https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/997#issuecomment-1664175213 also passed :+1:
Before merging, would you mind allowing me to run one more test ?  I'd like to validate a change to store all the files generated per arch on our side.

Before merging, would you mind allowing me to run one more test ? I'd like to validate a change to store all the files generated per arch on our side.

@tonyskapunk We are in no rush to merge this right now (soon, but not right now) so test away, we have an internal issue that QE also filed, and they probably won't re-test till tomorrow. Thanks for all the help on this!

Before merging, would you mind allowing me to run one more test ? I'd like to validate a change to store all the files generated per arch on our side.

@tonyskapunk We are in no rush to merge this right now (soon, but not right now) so test away, we have an internal issue that QE also filed, and they probably won't re-test till tomorrow. Thanks for all the help on this!

At this point we're completed validations on our side, and we will be ready when this change will be merged.
You may see some FAILURE messages coming from @dcibot and that is OK, it all depends how fast we merge our changes so the checked jobs start using them.
Thanks!

/lgtm

@bcrochet @acornett21 i tried running the code on my machine I got the following message:
>  ./preflight check container quay.io/gakrishn/test-multiarch:1.0 --pyxis-api-token <TOKEN> --certification-project-id <PROJECT-ID> --pyxis-host catalog.qa.redhat.com/api/containers --submit -d /run/user/1000/containers/auth.json
time="2023-08-08T19:14:00-04:00" level=info msg="certification library version" version="0.0.0 <commit: d3d26a96d46f99a89d8882ab7fc93c0337b01f56>"
Error: cannot process image manifest of different arch without platform override
Usage:
  preflight check container [flags]

Examples:
  preflight check container quay.io/repo-name/container-name:version

Flags:
      --certification-project-id string   Certification Project ID from connect.redhat.com/projects/{certification-project-id}/overview
                                          URL paramater. This value may differ from the PID on the overview page. (env: PFLT_CERTIFICATION_PROJECT_ID)
  -h, --help                              help for container
      --insecure                          Use insecure protocol for the registry. Default is False. Cannot be used with submit.
      --offline                           Intended to be used in disconnected environments and will tar artifacts used for submission,
                                          enabling other Red Hat managed tools/process to submit these artifacts at a later time.
                                          Cannot be used with submit.
      --platform string                   Architecture of image to pull. Defaults to runtime platform. (default "amd64")
      --pyxis-api-token string            API token for Pyxis authentication (env: PFLT_PYXIS_API_TOKEN)
      --pyxis-env string                  Env to use for Pyxis submissions. (default "prod")
      --pyxis-host string                 Host to use for Pyxis submissions. This will override Pyxis Env. Only set this if you know what you are doing.
                                          If you do set it, it should include just the host, and the URI path. (env: PFLT_PYXIS_HOST)
  -s, --submit                            submit check container results to Red Hat

Global Flags:
      --artifacts string       Where check-specific artifacts will be written. (env: PFLT_ARTIFACTS)
  -d, --docker-config string   Path to docker config.json file. This value is optional for publicly accessible images.
                               However, it is strongly encouraged for public Docker Hub images,
                               due to the rate limit imposed for unauthenticated requests. (env: PFLT_DOCKERCONFIG)
      --logfile string         Where the execution logfile will be written. (env: PFLT_LOGFILE)
      --loglevel string        The verbosity of the preflight tool itself. Ex. warn, debug, trace, info, error. (env: PFLT_LOGLEVEL)

2023/08/08 19:14:03 cannot process image manifest of different arch without platform override

❯ crane manifest quay.io/gakrishn/test-multiarch:1.0 | jq .
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:0b1e5207c31e005c1d986bb7466792bfd58c166bc2682241aae62740d77fbcd2",
    "size": 9575
  },
<snip>

That is not a multiarch manifest. That is a single arch manifest, and I'm guessing that since it is arm64, that it is also not the same as your runtime platform. Therefore, --platform override would be warranted here. That means it's working as designed.

coverage: 72.393% (-9.0%) from 81.388%
when pulling d781c79a14a09309c79d01caec984a90ad84740b on bcrochet:manifest-list
into 089e1b959136c4dbd3bb1862255481ec1b4cda38 on redhat-openshift-ecosystem:main.


Replace use of java.util.Properties in Bindings

Currently our variable storage, called "bindings", uses the java.util.Properties class for its implementation and its interface. It also pre-populates a variety of standard variables with newly instantiated objects WrappedCell, WrappedRow, and CellTuple which are rarely ever used.
Although it's strongly discouraged, the current code bypasses the setProperty and getProperty methods to use Hashtable's get() and put() directly, allowing it to support an effective contract of Map<String, Object> (but without support for null values since Hashtable doesn't support them).
Proposed solution

Add new interfaces in API which use Map<String, Object> in place of Properties for Evaluable2, HasFields2, HasFieldsList2, Function2, Control2, and Binder2 and deprecate the current interfaces.
Change the implementation to a subclass of HashMap<String, Object> which does lazy creation of all wrapped objects only when they are needed (named Bindings).
Function & Control registry will support implementations of both the new interface and the legacy interface and the evaluation machinery will choose the appropriate invocation based on type, preferring the new interface if it's available.
All bundled Functions & Controls will implement both the old & new interfaces using an abstract base class that includes a compatibility shim
Change code which does direct creation of bindings using new Properties() to use ExpressionUtils.createBindings(Project) like the main code does. Most occurrences of this pattern are scattered throughout the test code.
Log warnings on the console (or notify user at startup?) if any legacy Function, Control, or Binder is registered.

Alternatives considered

Not attempt compatibility and force all extension writers to update their code immediately.
Compatibility could be implemented on a per-call basis or the entire system could fall back to legacy mode if any legacy Function, Control, or Binder is registered - to be investigated if this simplifies things

Additional context

The Binder extension point is currently used by at least one extension (RDF Transform). If we want to get rid of it, it probably makes sense to offer an alternative.

The Binder extension point is currently used by at least one extension (RDF Transform).

Thanks for the heads up! I guess that illustrates the danger of assuming that any public interface is unused, even if it's not documented. I'll reintroduce the Binder2 interface that I had and figure out what needs to be done to preserve backward compatibility. I'm pretty sure there's no test coverage of this feature.
I've updated the description with more information on API compatibility.


update deepseek-coder to 128k

deepseek changes context from 32k to 128k

Thanks for trying aider and filing this issue. I've submitted a PR to litellm to update deekseek coder's context window.
https://github.com/BerriAI/litellm/pull/4541

I'm going to close this issue for now, but feel free to add a comment here and I will re-open or file a new issue any time.


Adding authenication and storing the data in cloud

We could add firebase backend to it so that when the user add the data it gets stored in the DB and can be retrived any time

@Alenjojo I am actually working on the backend with Node, Express and MongoDB Atlas. on the backend branch.

@Alenjojo I am actually working on the backend with Node, Express and MongoDB Atlas. on the backend branch.

Oh okay cool

Oh okay cool


Do not add newline before inline css comments when formatting css files

Before formatting:

After formatting:

Do not break line with inline comments.

We use beautifyjs as the underlying CSS formatter. Would you mind filing this request directly against https://github.com/beautify-web/js-beautify ?


Storage fixes for new format

-- Cash fix merged from https://github.com/twintproject/twint/issues/946#issuecomment-706596925
-- Pandas storage fix: stop accessing removed fields (same as cash fix) + handle new stringly-typed datetime format
A weirdness worked around here is the new tweet format timestamps are str instead of datetime/epoch_int, so to fix pandas parsing, exported the str format to make reusable. IMO better to get back to having normalized types, but this should help for interim.

so theses fields are not related to cashtags, this will break the library again. might need to test this. Iam not sure what these fields are for, as I never got to see the older version of twitter scraping result which originally contained these fields. So, I think keep them commented out for now.

Yes, previous to the api change, twint was able to get reweet info, but the api fix meant we don't get them anymore. Commenting out seems legit, had to do same for pandas.
This is a breaking api change for downstream clients. It's a pretty core data gap, so ok IMO.


Server injected Modal not executing embedded JavasScript

My apps display modal forms using HTML injected from the server. The JavaScript in Boostrap 4.0 and above no longer gets executed when the modal is displayed.
The issue is in the modals _proto._showElement method and the following line:
document.body.appendChild(this._element);
For JavaScript to work again it needs to be replaced with something like:
$("body").append(this._element);

Bug reports must include a live demo of the problem. Per our contributing guidelines, please create a reduced test case via CodePen or JS Bin and report back with your link, Bootstrap version, and specific browser and OS details.

Hey there!
We're automatically closing this issue since the original poster (or another commenter) hasn't yet responded to the question or request made to them 14 days ago. We therefore assume that the user has lost interest or resolved the problem on their own. Closed issues that remain inactive for a long period may get automatically locked.
Don't worry though; if this is in error, let us know with a comment and we'll be happy to reopen the issue.
Thanks!
(Please note that this is an automated comment.)


Automated cherry pick of #3005: fix: tenant filter fail to work

Cherry pick of #3005 on release/2.11.
#3005: fix: tenant filter fail to work

/approve
/lgtm

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.11@127a54e). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             release/2.11   #3007   +/-   ##
==============================================
  Coverage                ?   7.16%
==============================================
  Files                   ?     476
  Lines                   ?   76636
  Branches                ?       0
==============================================
  Hits                    ?    5494
  Misses                  ?   70679
  Partials                ?     463

Flag
Coverage Δ

#aFlag
7.16% <ø> (?)

Impacted Files
Coverage Δ

pkg/cloudcommon/db/fetch.go
0% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 127a54e...fd9154d. Read the comment docs.


tests: Reenable disabled legacy tests in ParallelEoSStreamProcessorTest

To either implement or delete
Checklist

[ ] Documentation (if applicable)
[ ] Changelog

was closed by accident

Closing - Stale.