vlang: weekly.2023.44 -> 0.4.3

Description of changes
Can track version releases from now on rather than weekly.
https://github.com/vlang/v/releases/tag/0.4.3

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Opening this PR for https://github.com/NixOS/nixpkgs/pull/270941#discussion_r1415598046

P.s. I wanted to ask your opinion on rebasing from gcc to tcc? Would this be desirable? If so I can take a look at the code (for a later PR).

P.s. I wanted to ask your opinion on rebasing from gcc to tcc? Would this be desirable? If so I can take a look at the code (for a later PR).

If you're talking about the wrapper, I think we should stick to gcc for the optimizations.
We really need vlang-unwrapped so we can have a wrapper like this for default and and the option of having the regular unwrapped executables untouched.


[ALLUXIO-1915] Enable support for reading UFS files of unknown length

https://alluxio.atlassian.net/browse/ALLUXIO-1915

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9516/
Test PASSed.

Merged build finished. Test PASSed.

LGTM. Thanks for adding this feature.

LGTM


Validation of Font mimeTypes

Bug Report
Currently the validation of font mimeTypes is off, especially for woff2 files. The only way to upload these files is to completely omit the mimeTypes array, allowing any fileType to be uploaded (not recommended).
Steps to Reproduce
I tried different fontTypes in the mimeTypes array:
mimeTypes: ['application/font-woff'], -> works for .woff files
mimeTypes: ['font/woff'], -> doesn't work for .woff files
mimeTypes: ['application/font-woff2'], -> doesn't work for .woff2 files
mimeTypes: ['font/woff2'], -> doesn't work for .woff2 files
mimeTypes: ['font/*'], -> doesn't work for any font files
Other Details
Payload Version 1.5.9

Hey @owieth — unfortunately there's nothing that we can do about this with Payload itself.
I just tested a few woff2 files with this site:
https://www.htmlstrip.com/mime-file-type-checker
They all came back as application/octet-stream which is unspecific. I am betting that you would need to whitelist the application/octet-stream type if you want to upload woff2 files.
We only use a simple mime type checker so there is no good way for Payload to handle this in our own code, unfortunately. Manually watching for .woff2 extensions and setting mime type based on file extension would not be ideal and would also be applicable for many, many other filetypes as well.
Does this make sense?

Hey @owieth — unfortunately there's nothing that we can do about this with Payload itself.
I just tested a few woff2 files with this site: https://www.htmlstrip.com/mime-file-type-checker
They all came back as application/octet-stream which is unspecific. I am betting that you would need to whitelist the application/octet-stream type if you want to upload woff2 files.
We only use a simple mime type checker so there is no good way for Payload to handle this in our own code, unfortunately. Manually watching for .woff2 extensions and setting mime type based on file extension would not be ideal and would also be applicable for many, many other filetypes as well.
Does this make sense?

Sup @jmikrut
Thanks for checking, that's a bummer but I will work with application/octet-stream 👀


Pulling the current branch does NOT update the working tree

HEAD gets updated, but not working directory or index (so as with git reset --soft), and this is not what user would expect from a pull-like action. This is probably the result of how git-fetching the current branch operates.

Probably related to this (effect of this issue): When I was on root branch (if it even is important) I invoked pull action from toolbar and this rather something like fetch than pull. When I repeated pull action (this time from context menu) then true pull was done.

Your comment @micpiotrowski probably refers to an older version of the plugin as for some time there's no longer any pull action in the left-side toolbar. Anyway, it might return to toolbar once #427 is addressed. Anyway, let's first address the original issue with the  working tree first.

We could use the Pull Dialog for the current branch case:

@mkondratek would be nice if the pull operation we provide is fast-forward-only (at least by default) - will it be possible with the above dialog?

whoops, the checkbox that we can see above No fast forward allows switching between the default --ff and --no-ff
furthermore, the dialog will possibly change with the 202.* :confused:
A look from 2020.2 BETA:

Heh, looks that JB is basically changing the existing git4idea dialogs into "command line builders" xD

My investigation has proven that the dialog will actually change. Hence we should not depend on it.
Reimplementation (less stable solution, more work) is not necessary since we do know all the options & params to perform pull --ff-only on the current branch.
Ofc we could disable the pull for the current branch (some solution, stable & easy).
Going a little lower we can use GitLineHandler (more stable solution, not so hard).
Even though the logic around it seems to be a bit complex I think that this is the best option.
The draft for the third one: #438


Scripts for creating vocabulary files automatically from Wikipedia

Made an automatic workflow for creating vocabulary files from Wikipedia. Thought, might be useful for other folks as well.

Thanks a lot for contributing this – I'm sure this will be super useful for others!
Btw, for spaCy 2.0 we were thinking about adding another CLI command spacy convert (or something similar) that includes some of the most common converters for spaCy's formats and some other model utilities. So maybe there's a way to include a simple version of your script in there as well?


How to use Regular Expressions with Google Analytics Reporting API V4

I thought I would give this a try. Before I start, I would like to say that I have already posted on StackOverflow: https://stackoverflow.com/questions/46960709/how-to-use-regular-expressions-with-google-analytics-reporting-api-v4 I have also done various searches on Google Search without any success. I am unable to use REGEX. Any help is greatly appreciated.

It looks like this is being answered on SO.

Hello Matt, yes someone is trying to help me though we still haven't arrived on a plausible solution. At this point, I am ditching the PHP client altogether.


fix support create dup table without key

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


add Issif as maintainer

Signed-off-by: Issif<EMAIL_ADDRESS>
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface

/kind content

/kind translation

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog

/area documentation

/area videos

What this PR does / why we need it:
This PR adds Issif as maintainer for this repo. Following our governance, the suggestion has been made in community call of March, 3rd and all present member agreed.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Issif
To complete the pull request process, please assign kris-nova after the PR has been reviewed.
You can assign the PR to them by writing /assign @kris-nova in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cc @radhikapc @Rajakavitha1

LGTM label has been added.  Git tree hash: ab8c29daacc479a1e9bce0ff6c6ee3d119140f84

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Issif, mstemm
To complete the pull request process, please assign kris-nova after the PR has been reviewed.
You can assign the PR to them by writing /assign @kris-nova in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


.\lean_packages\std..\Std\Data\Nat\Lemmas.lean:152:42: error: no goals to be solved?

I'm trying to update lean4-samples to latest mathlib4 so I can pick up the new symm tactic, but the Std library build fails with:
error: > LEAN_PATH=.\build\lib;.\lean_packages\mathlib\build\lib;.\lean_packages\std\build\lib;.\lean_packages\Qq\build\lib c:\Users\clovett\.elan\toolchains\leanprover--lean4---nightly\bin\lean.exe -DwarningAsError=true -Dlinter.missingDocs=true .\lean_packages\std\.\.\Std\Data\Nat\Lemmas.lean -R .\lean_packages\std\.\. -o .\lean_packages\std\build\lib\Std\Data\Nat\Lemmas.olean -i .\lean_packages\std\build\lib\Std\Data\Nat\Lemmas.ilean -c .\lean_packages\std\build\ir\Std\Data\Nat\Lemmas.c
error: stdout:
.\lean_packages\std\.\.\Std\Data\Nat\Lemmas.lean:152:42: error: no goals to be solved
.\lean_packages\std\.\.\Std\Data\Nat\Lemmas.lean:157:26: error: application type mismatch

This is usually a lean version mismatch. Double check that the version in your lean-toolchain matches the version in the lean-toolchain of mathlib (currently leanprover/lean4:nightly-2022-10-20).

Ah, that was it thanks.


Build gRPC samples with Scala 2.13 when upstream deps are ready for 2.13

This is a spin-off from #32.
gRPC samples can't build with 2.13 for the moment.

Pretty sure we can close this, both Scala and Java grpc example run and are tested in CI.


Streams optimization

This is proof of concept of significant StreamReader speedup. The main idea is avoiding memory copying and reallocations. The collections.deque is used instead of the bytearray, but unlike for DataQueue, no reading limitations are applied.
Performance
My benchmarks: https://gist.github.com/homm/3b9c2909905f76668bf9

Test
old time
new time
speedup

Large chunk

test_stream_feed
0.0298
0.00001
2450x

test_stream_read_all
0.2127
0.00005
3948x

test_stream_read_less
0.2433
0.0506
4.8x

test_stream_read_any
0.0608
0.00005
1128x

test_stream_read_exactly
0.2005
0.0173
11.5x

Small chunks

test_stream_feed
0.0674
0.0094
7.1x

test_stream_read_all
0.2633
0.0508
5.2x

test_stream_read_less
0.2165
0.0556
3.9x

test_stream_read_any
0.1022
0.0237
4.3x

test_stream_read_exactly
0.1825
0.0371
4.9x

Memory
As a result of removing relocations, the memory consumption and fragmentation is also greatly reduced. I've got following result for the benchmark:

Without test_stream_read_all test.

Old: mostly 350-450MB, up to 550MB
New: 92MB

With test_stream_read_all test.

Old: mostly 350-450MB, up to 550MB
New: 170MB

Distinct
The main difference is how read(BIG_LIMIT) and readany() produce the result when the buffer is full: In the old implementation, the whole buffer is returned at once. In the new implementation, only the first chunk is returned.
Roadmap

readline() is not rewritten.
Most tests are failed because they are relying on the internal StreamReader implementation.

Should I go on?

Sure! Please re-write readline(), do not touch tests, I'd like to test it on real system

@fafhrd91 Done.
In my opinion readline() is totaly broken, because it checks limits after reading from _buffer and also it corrupts input stream if line size > limit, because it reads the undefined amount of data and lose it. But I think this is not the topic of current PR, and I've restored the old behavior.

Thank you a lot @homm, I'm pretty excited to test that with the FrameworkBenchmarks suite to see the difference with the previous version of aiohttp.
Can I already test with this branch, or I must wait that you fix all the tests compliance first ?

@GMLudo should be fine without tests

@homm Nice work! In additional to aiohttp test, it also breaks aiocouchdb ones:
Traceback (most recent call last):
  File "/home/kxepal/projects/aio-libs/aiocouchdb/aiocouchdb/tests/utils.py", line 36, in wrapper
    return testcase.loop.run_until_complete(future)
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/base_events.py", line 316, in run_until_complete
    return future.result()
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/tasks.py", line 238, in _step
    result = next(coro)
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/tasks.py", line 377, in wait_for
    return fut.result()
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/tasks.py", line 236, in _step
    result = coro.send(value)
  File "/home/kxepal/projects/aio-libs/aiocouchdb/aiocouchdb/v1/tests/test_document.py", line 232, in test_get_with_atts
    yield from result.release()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/multipart.py", line 187, in release
    yield from self.resp.release()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/client_reqrep.py", line 638, in release
    chunk = yield from content.readany()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/streams.py", line 391, in wrapper
    result = yield from func(self, *args, **kw)
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/streams.py", line 454, in readany
    return (yield from super().readany())
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/streams.py", line 199, in readany
    return self._read_nowait()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/streams.py", line 253, in _read_nowait
    self._buffer_size -= len(data)
TypeError: object of type 'int' has no len()

I tried to reproduce the problem with public instance, but here I also found that patch breaks multipart reader as well in other way:
import asyncio
import aiohttp

@asyncio.coroutine
def go():
    resp = yield from aiohttp.request(
        'get',
        'https://kxepal.cloudant.com/aiohttp-496-bug/docid',
        headers={'Accept': 'multipart/related'},
        params={'attachments': 'true'})
    reader = aiohttp.MultipartReader.from_response(resp)
    while True:
        part = yield from reader.next()
        if part is None:
            break
        print((yield from part.read()))

if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(go())

Expected:
bytearray(b'{"_id":"docid","_rev":"6-783d465b150e111b56447cd1d4e533a8","_attachments":{"test":{"content_type":"application/octet-stream","revpos":1,"digest":"md5-JS3VMT0Dk7FXE0dUvc9Vag==","length":15,"follows":true}}}')
bytearray(b'Hello, aiohttp!')

Actual:
Traceback (most recent call last):
  File "/home/kxepal/projects/aio-libs/aiocouchdb/test.py", line 21, in <module>
    asyncio.get_event_loop().run_until_complete(go())
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/base_events.py", line 316, in run_until_complete
    return future.result()
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/tasks.py", line 236, in _step
    result = coro.send(value)
  File "/home/kxepal/projects/aio-libs/aiocouchdb/test.py", line 14, in go
    part = yield from reader.next()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/multipart.py", line 178, in next
    item = yield from self.stream.next()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/multipart.py", line 460, in next
    self._last_part = yield from self.fetch_next_part()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/multipart.py", line 475, in fetch_next_part
    headers = yield from self._read_headers()
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/multipart.py", line 536, in _read_headers
    headers, *_ = parser.parse_headers(lines)
  File "/home/kxepal/projects/aio-libs/aiohttp/aiohttp/protocol.py", line 85, in parse_headers
    raise errors.InvalidHeader(name)
aiohttp.errors.InvalidHeader: 400, message='Invalid HTTP Header: 50E111B56447CD1D4E533A8","_ATTACHMENTS"'

parser.parse_headers receives the following lines:
['', 'content-type: application/json\r\n\r\n{"_id":"docid","_rev":"6-783d465b1', '50e111b56447cd1d4e533a8","_attachments":{"test":{"content_type":"application/octet-stream","revpos":1,"digest":"md5-JS3VMT0Dk7FXE0dUvc9Vag==","length":15,"follows":true}}}\r\n--c1782ac8c1a68da8b0bd4f8983b86fa3', '']

while it should stop right after '\r\n\r\n'.
Thoughts? If you need any additional information, feel free to ask.

@kxepal Thanks, I've fixed the error in readline(), so MultipartReader works again.
But I can't reproduce the error with aiocouchdb. I've ran tests from master and got a lot exceptions in __del__ method of client_reqrep.ClientResponse (I have Python 3.4):
Traceback (most recent call last):
  File "/Users/Master/Code/async/aiohttp/aiohttp/client_reqrep.py", line 549, in __del__
    self._loop.call_exception_handler(context)
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception ignored in: <bound method HttpResponse.__del__ of <ClientResponse() [200 None]>
{'CONTENT-TYPE': 'application/json'}

But except for these errors all tests are passed.
Ran 302 tests in 2.083s
OK (skipped=9)

@homm Oh, my. It seems I didn't push the fixes. Thanks for notice!
The problem happens with make check-couchdb - simple make check runs tests against mocks and cannot || not able to trigger this issue. However, I found the error on my side. Ironically, it's within the danger zone where I touch _buffer that you had changed (: s/extend/append/ solves the problem.
Sorry for false alarm (:

@kxepal

s/extend/append/ solves the problem.

Unfortunately no, because there is also _buffer_size and total_bytes properties. Why you can't use feed_data there?
I had the idea to create a wrapper around collections.deque with bytearray syntax (extend and len) for backward compatibility. But I don't know Is it really necessary for private property of 0.x version library :—)

@homm
Actually, I don't know why. Strange decisions eventually happens (: Will investigate public API for better workaround.
As for idea to mimic deque to bytesarray, I think it doesn't worth any effort. In the end, field is private - there are no guarantees provided on them.

@kxepal

The problem happens with make check-couchdb

After that I have various exceptions from new master:
  File "/Users/Master/Code/async/aiohttp/aiohttp/connector.py", line 299, in connect
    'Cannot connect to host %s:%s ssl:%s' % key) from exc
nose.proxy.ClientOSError: Cannot connect to host localhost:5984 ssl:False
----------------------------------------------------------------------
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/selectors.py", line 451, in __init__
    self._kqueue = select.kqueue()
OSError: [Errno 24] Too many open files

----------------------------------------------------------------------
Ran 305 tests in 0.621s

FAILED (SKIP=12, errors=255)

@homm hm...what's your fd limit?

@homm i tested your patch in production, everything is good. we need to fix kxepal's tracebacks and update tests, then lets merge change.
regarding performance, in system with normal load, small/fast requests, mostly requests to external subsystems difference is not visible. but that is expected. in system with a lot of load (file upload/download) performance slightly better, but memory usage is much better.

ping

Sorry. I'll try to find some time for this tomorrow.

Ok, done.
Instead of checking internal stream._buffer, the stream is consumed until the end. If there was not eof, it is added right before reading.

\o/ it's green! +1

Thanks!


Added Anthropic Claude

Added Anthropic's Claude v1.3 and Claude-Instant v1.0
I am currently testing locally to check the metrics (sometimes the model chats too much a bit like ChatGPT which causes bad results). i will update this once the tests are done.

Changes made.
I also changed the tokenizer

Merge conflicts resolved

Changes made. I raised new exceptions. COuld you handle them in a separate PR?


Implement a way to choose a test case in the interface

It should be possible to change which test case the solution is tested against. It would also be beneficial to show a diff between the output and expected output.

this is done now (b4e3a3f839db79934ef617ccb849fee05590e3dc)


Add TOML config for enabling JMX for a datasource

Description:
This section [1] needs to be updated with the toml config with jmxEnabled=true etc. as it currently has XML configs.
[1] https://apim.docs.wso2.com/en/latest/administer/logging-and-monitoring/monitoring/jmx-based-monitoring/#enabling-jmx-for-a-datasource

Uvindra said that the templating is as follows:
"database.shared_db.jmx_enable": "false"

So you would just have jmx_enable = "false" under the appropriate DB config in deployment.toml
Example under [database.shared_db] or [database.am_db]
[database.shared_db]
jmx_enable = "false"

Fixed [1]
[1] https://apim.docs.wso2.com/en/latest/observe/api-manager/metrics/jmx-based-monitoring/#enabling-jmx-for-a-datasource


3rd party container registry integration

Hello Kubescape team,
I received an email outlining some contribution areas for Kubescape. I specifically saw an ask for integrations with 3rd party container registry APIs and vulnerability scan results.
Recently I have been doing a lot of development work with the Harbor container registry https://goharbor.io, specifically utilizing their APIs to create tools which trigger and analyze vulnerability scan results.
I have a few ideas for integrations that may be useful, however, I am not 100% sure what kinds of integrations you are looking for specifically.
Triggering image scans and retrieving scan results are not unique to Harbor, it is also available in several other container registries. Perhaps this could be a starting point to outline an interface for other contributors to add support for other registries?
These are the initial ideas I had for integrations which I feel could be useful. Results would be in JSON and used for either local consumption or submission to armo cloud.
Registry Login

kubescape registry login https://myharborregistry.com

Trigger a vulnerability scan of a single image (or all images) and/or retrieve the scan results

kubescape registry scan  --images https://myharborregistry.com/project/repo/image@sha256xxxxxxx
kubescape registry scan all

Search registry images for specified vulnerabilities and retrieve a report of images that contain them

kubescape registry search --cves CVE-2021-44228,CVE-2021-25742

Generate an image vulnerability report for images in their harbor instance. Show CVSS score statistics for images. Similar to the Kubescape scan results, but for image(s) CVE's

kubescape registry generate-summary

Please let me know if these types of integrations could be useful, or if the team had other ideas as to how 3rd party container registries could be utilized. Happy to help where I can. Thanks!

Hi @tks98
You made an impressive discussion starter :)
First, I'd like to give an answer of "why" to integrate Kubescape with container registries.

We are planning to create controls which take into account image vulnerabilities, example: looking for public internet facing workloads with critical vulnerabilities. These are seriously effecting the security health of a cluster and therefore we think it is important to cover it. We think that most container registries are/will support image scanning like Harbor and therefore the ability to get information from them is important.
There are information in the image repository which is important for existing controls as well. They are incomplete without it, example see this issue: #19 . These are not necessarily image vulnerability related. Can be information in the image manifest (like the issue before), but it can be the image BOM related.

Perhaps this could be a starting point to outline an interface for other contributors to add support for other registries?

You have a valid point about interoperation and extensibility. At the first step, we would like to define this interface.
I suggest the interface should cover the following high level functionality:

Authentication against the information source (abstracted login)
Triggering image scan
Reading image scan status (with last scan date and etc.)
Getting vulnerability information for a given image
Getting images for a given vulnerability
Getting image information (like manifest)

The implementation of this interface should be able to work with other implementations, since Kubescape will see multiple image sources in a single cluster.
Please comment. (you can also find us on our Discord channel, if you want more interactive responses :) )
@Bezbran @dwertent , we will need to think how will we connect OPA/Rego-s to this interface.
Cheers

Thanks for the quick reply @slashben! I think the high level functionality outlined here makes a lot of sense for 3rd party registry integration. I'll message the discord with further questions as I start implementing these for harbor.

@tks98
Please check out https://github.com/armosec/kubescape/blob/dev/docs/proposals/container-image-vulnerability-adaptor.md
Comments are welcome!

@tks98
We have done some progress and developing the first adaptor implementation. It is still in our own branches but it will be published in dev branch by the end of week.
(preview: https://github.com/slashben/kubescape/tree/master/container-images)

@slashben Thanks for the update! Once the adaptor implementation is finalized I can adjust my harbor adaptor to fit that spec. So far have the ability to authenticate, trigger scans, and receive vulnerability information for images in the harbor registry.

We have decided to remove the adaptors from Kubescape.
Kubescape will support scanning the image directly.


Redis

General Troubleshooting

[X] I have checked for similar issues on the Issue-tracker.
[X] I have updated to the latest JDA version
[X] I have checked the branches or the maintainers' PRs for upcoming features.

Feature Request
I would like to request a new feature that allows us to store cached member/user/guild/... on external providers like Redis. Currently, these caches are stored in memory, which can be a challenge for larger servers or applications with a high number of users and data.
Example Use-Case
No response

We currently have no plans to rewrite the internal cache components.


Reorder Codes

Is it possible to change the order of emoji codes from complex to simple? Or change the map generation in https://github.com/iamcal/php-emoji/ according to the problem?
Explanation: in the current order the php-emoji, js-emoji and maybe other scripts convert the first found code. That is, for complex emojis, this is only part of the entire emoji.
Example:
👩👩👦👦 (woman-woman-boy-boy)
Part of the PHP code:
...	"\xf0\x9f\x91\xa9\xe2\x80\x8d\xf0\x9f\x91\xa9\xe2\x80\x8d\xf0\x9f\x91\xa6"=>"1f469200d1f469200d1f466",
"\xf0\x9f\x91\xa9\xe2\x80\x8d\xf0\x9f\x91\xa9\xe2\x80\x8d\xf0\x9f\x91\xa6\xe2\x80\x8d\xf0\x9f\x91\xa6"=>"1f469200d1f469200d1f466200d1f466",
...
With the current order of codes, conversion will occur only on the first occurrence. Thus, the emoji is divided into 2: woman-woman-boy and boy.

that's a bug in php-emoji ; it should sort the map to have longer matches first to avoid this. closing the bug here, since we always use lexical codepoint order sort here for consistency


remove_compound_assignment copying comments/newlines in between statements

How to replicate:

Go to https://darklua.com/try-it/
Use the following rule below:

{
  rules: [
    'remove_compound_assignment',
  ],
}

Results:

Input
Output
Expected

local x = 0
x += 1-- this is ok

local y = 0
print(y)
y += 1-- this is ok

local y = 0

y += 1-- one newline appears

local z = 0
--duplicate
z += 1--it copies the comments

local a = 0
--copies
--all
--comments

--in between
a += 1

local x = 0
x =x +1-- this is ok

local y = 0
print(y)
y =y +1-- this is ok

local y = 0

y =
y +1-- one newline appears

local z = 0
--duplicate
z =--duplicate
z +1--it copies the comments

local a = 0
--copies
--all
--comments

--in between
a =--copies
--all
--comments

--in between
a +1

local x = 0
x =x +1-- this is ok

local y = 0
print(y)
y =y +1-- this is ok

local y = 0

y =y +1-- one newline appears

local z = 0
--duplicate
z =z +1--it copies the comments

local a = 0
--copies
--all
--comments

--in between
a =a +1

Thank you for the detailed report, I can already see the fix for this issue. The rule is cloning the variable with all its token information. If it just cleared the tokens it'd probably fix all of this 🚀


[AIRFLOW-1906] Remove ccs-kerberos from dependency

Make sure you have checked all steps below.
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-1906

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

we have got both pykerberos and kerberos are dependencies. We should be using only one.
requests_kerberos is using pykerberos.
https://github.com/requests/requests-kerberos/issues/97
https://github.com/requests/requests-kerberos/issues/63
https://github.com/requests/requests-kerberos/pull/106
Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

[x] Passes git diff upstream/master -u -- "*.py" | flake8 --diff

Codecov Report

Merging #2865 into master will decrease coverage by 0.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2865      +/-   ##
==========================================
- Coverage   74.05%   73.96%   -0.09%
==========================================
  Files         160      160
  Lines       12139    12139
==========================================
- Hits         8989     8979      -10
- Misses       3150     3160      +10

Impacted Files
Coverage Δ

airflow/task_runner/bash_task_runner.py
93.33% <0%> (-6.67%)
:arrow_down:

airflow/utils/helpers.py
53.44% <0%> (-2.88%)
:arrow_down:

airflow/jobs.py
79.44% <0%> (-0.45%)
:arrow_down:

airflow/models.py
87.47% <0%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9731ce6...be58a3e. Read the comment docs.


proposal: enhance import directive to allow for alternatives on failure to better support promotion of experimental packages into the standard library

This issue is a generic enhancement request for go.
Background
An experimental package called "golang.org/x/net/context" exists.  This package was brought into the standard language in go 1.7 as "context".  This is one example - it will happen again in the future many times over.
Problem
The import directive does not allow for alternatives or versioning to facilitate a smooth transition between experimental and official standard package status.  While go is backwards compatible, this transition is not, and it has impacted many projects.
Proposal
Modify the import directive in the language to allow for a fallback position, for example:
ImportDecl       = "import" ( ImportSpec | "(" { ImportSpec ";" } ")" ) .
ImportSpec       = [ "." | PackageName ] ImportItems .
ImportItems      = ImportPath [ [ "or" ImportPath ] ... [ "silently" ] ] .
ImportPath       = string_lit .

for example:

    import "context" or "golang.org/x/net/context" silently

The two packages in this case have compatible APIs and therefore this single line of code would have spared hundreds of projects from having to deal with the transition of the context package into official status.   The silently keyword would omit any warnings; without the keyword a warning would be emitted when an alternative is loaded.
Proof of Problem

https://github.com/golang/mock/issues/116 (resolution: please update your go)
https://github.com/golang/mock/pull/118 (resolution: switched back to golang.org/x/net/context)
https://github.com/grpc/grpc-go/issues/711
https://github.com/golang/go/issues/17040 (tool proposal to help people move forward)
https://github.com/golang/protobuf/issues/217 (discuss when to make the switch)
https://github.com/go-kit/kit/issues/420
https://github.com/labstack/echo/issues/899
https://github.com/golang/go/issues/15471
The Apache Thrift project detects the go version at build time and inserts different files into the build to support <1.6 or >1.7 based solely on this package and imports "the right one".

There are many other examples of this.

I've marked this a proposal but you'll definitely need to clarify the exact semantic of the change you are proposing to make this report a serious proposal.
Also, Go tools almost never emit "warning", so I'm not convinced that silently modifier will be well received.

Since the concern is mainly for packages brought into the standard library, I think you need to explain why build tags are insufficient.
For example, for the context package, as long as you want to support versions of Go before 1.7, import "golang.org/x/net/context".  That package is written to use build tags to make the right thing happen: if using Go 1.7 or later, you will in effect get the standard library "context" package.

Closing in favor of build tags and the ongoing package versioning work (#24301).


parse single field

I have form with conditional updates and sometimes i need to serialize only one field.
i tried:
$('#' + elm_id).serializeJSON();
and
$('#edit_user #' + elm_id).serializeJSON();
both wit try by class name too..
it works in JS console but not in the code:
var elm = $('#' + elm_id)
console.log('ELM', elm); // return jquery object
var payload = elm.serializeJSON();
console.log('PAYLOAD', payload); // return {}

above works when i run it manually.
this code run on document ready and behind other function

OH !
sorry i was disabling input before reading serializeJSON() :)
just moved prop disable after serialize :)
Thank you for great library!


chore: fix some typos

IMPORTANT: Please do not create a Pull Request without creating an issue first.Any change needs to be discussed before proceeding.
Closing issues

closes #ISSUE_NUMBER_GOES_HERE
...
Description
fix some typos

...
Changes

High level
changes that
you made
...

Thank you @depthlending!


[BUG]

🐛 Describe the bug
The modal component doesn't respect the Dark Mode theme
🧐 To Reproduce
Steps to reproduce the behavior:

Go to this page in the nimbus docs
Set Dark Mode

✅ Expected behavior
The modal should respect Dark Mode.
📷 Screenshots

🖥 Desktop (please complete the following information):

OS: macOS 13.5.2
Browser: Chrome
Version: 117.0.5938.88

@santigandolfo muito obrigado por reportar o bug foi corrigo na #203.


NEW FUNCTION: [REDACTED]

A new function is being developed, and it's a secret!

Secret function appears to be working end-to-end now. Undergoing unit testing...

This new function is Playlist Manager!
It is available in BETA as of v2<IP_ADDRESS>0


Simplify buffer interface greatly

AggBuf shouldn't be that complicated. Advancing should split the buffer, always, and return an AggSlice (maybe those should be renamed AggMut and Agg). AggSlice should not hold a Rc to AggBuf / AggBufInner, it should have a SmallVec<[Buf; 1]> and that's it.
This should remove the need for borrow_mut in a bunch of places.

This actually might make parsing harder, so I'm not so sure about it anymore..
In the current system, AggSlices have an index into the initialized portion of the original AggBuf, which allows slicing it so we can get an AggBuf of the "rest" (part not used when parsing). If AggSlice were a SmallVec of Buf, we'd need to keep track of that index separately.
Maybe it's still worth it?


Setup GitHub Pages

This sets up GitHub Pages under /docs allowing contributors to improve documentations.
Many documentations are outdated or hard to read and needs major rewrite, but this PR will try to focus on setting up GitHub Pages and improve documentation structure.
I'm going to make a POC repository for reviewing and commit to this repository once it's done.
Any feedback is appreciated.
Features

Mostly markdown (kramdown with GFM parser) for easier contribution.
Makes maintenance easy by taking advantage of templates, data files, and YAML front matter.
Documentations are organized by topic and target audience instead of simply migrating the existing documentation.

Overview/Screenshots
Using Data File
Some tables are setup so updates in data files will be reflected.

Using Jekyll Collections and YAML Front Matter
This is similar to using data files, but data for the following table is extracted from YAML front matter in markdown. The content after the front matter is used to generate a page. For example, adding a new file under /_environments will add a new entry in the table and generate its page in /environments/:language.

Improved Documentation Structure
Documentations are organized by topic and target audience. This helps both new and experienced users. Please let me know what you think.
Currently grouped like this:
about
Introductions to Codewars platform, mostly for new users.

Getting started
What is ...?
Key concepts
Terminology
Community

references
Simple reference manuals intended for things like looking up available test methods. Documentations under this group needs to be skimmable and consistent without noise.
tutorials
Documentations for teaching users.

How to ...?
Best practices

snippets
Commonly used patterns.

Recipes/Cookbooks (Problem, Solution, Discussion)
Examples
Snippets
Useful patterns

An example of recipe/cookbook style documentation:

TODO

[ ] License
[x] Navigation (Do this in separate PR)
[x] Styling (Do this in separate PR)
[x] Documentation for contributors (Do this in separate PR)

Closes: #881, #845, #846, #891, #345

@kazk Awesome outline and overall game plan. The structure you laid out looks great to me, and I love the idea of having Recipes/Cookbooks. That will be a massive help to the community.
Something else to consider for when this is up and running is which parts of the codex that @bkeas built should be incorporated.

Something else to consider for when this is up and running is which parts of the codex that @bkeas built should be incorporated.

You meant @bkaestner - which parts of your Codex do you think is crucial to include in our new CW documentation?  Any feedback would be much appreciated :smile:
By the way @jhoffner a few points that I (personally) consider important to include in our new Docs from the Codex just by skimming through it:

2.3 Follow your language's naming conventions - an Issue I raised about language naming conventions not followed in JavaScript I raised a few days back got prematurely resolved by another Moderator with the reason of "the Kata Best Practices doesn't mention naming conventions specifically".  However, if I remember correctly, you mentioned that following naming conventions is an official CW stance?
2.6 Use proper formatting.  Sometimes, new Codewars Kata authors may not be familiar with Markdown (or are just too lazy to use it) and end up writing hard-to-read code examples written in plaintext which is a bit annoying.
2.8 Use the preloaded section only for helpers and constraints - some new Kata authors may not know that it is not safe to put their reference solution in the Preloaded section so it is worth a brief mention
This particular section of the paragraph directly below Major Heading 4 (Tested testing theories): "Make sure to file an issue if you notice that a kata is lacking some tests. This will prevent a moderator from approving the kata. This will increase the overall quality of the approved katas."  I see way too many users upvoting a Kata with appalling quality (e.g. this one prior to the Kata author reluctantly editing the Kata to include different inputs and basic random tests - its satisfaction rate in my opinion should be zero if everyone is doing their job properly in assessing this Kata)
4.1 Consider writing your tests first - not that important but maybe worth a quick mention (just a sentence or two)
4.2 on never using Test.expect without a suitable error message upon failure.  Just a brief mention would suffice
4.5 Use random tests - Definitely.  4.6 for emphasis is optional.
4.7 Hide your solution.
4.9 Handle floating point values the right way.  Yesssss.  Luckily in JavaScript we now have Test.assert(Not)ApproxEquals but for other languages perhaps a basic code example of what a floating point assertion should look like could be included.
4.11.1 is perhaps worth a brief mention.  Just not very long ago, I was the type of person to accept any incoming translations, no questions asked, which naturally lead to some faulty translations being approved and users raising Issues repeatedly on certain languages (but I can't fix them).  Now I know better :wink:
4.11.2 Handling merge conflicts.  Good to include in our new Docs as I was previously unsure how to handle them and had to learn how to resolve them through trial and error.
5.3 Language Remarks (Java) - one of the Java translations I previously accepted to my Interpreter Kata Series had the actual and expected values swapped which lead to 4 Issues being raised on my Kata at one time.  I had to explain that I had no knowledge of the JUnit testing framework and thought the translation looked perfectly fine to me at the time before the user agreed to resolve the Issue on my Kata.

That being said, I find the entire Codewars Codex valuable so perhaps our new Docs could have a "User Contributed Documentation" section (or similar) and just link to the Codex so users can refer to the entire Codex if necessary.

[ ] Documentation for contributors

You should probably discuss the licence for documentation too.

Something else to consider for when this is up and running is which parts of the codex that @bkeas built should be incorporated.

Small remark: the Codex is CC-BY-SA-4.0. Just in case someone was wondering.

This is little off topic, but I think the help in the kata editor needs to be improved.

This might be the reason for new authors using Test.expect. I don't think "Click here" is clicked by those who need to click :(

This might be the reason for new authors using Test.expect. I don't think "Click here" is clicked by those who need to click :(

No wonder!  I've never had to refer to the "Help" tab myself, even when I was new to Codewars.  I always referred to the entire Test Reference.  I agree with @kazk - that small paragraph about Test.expect should be removed altogether so new Kata authors are forced to click on the link containing the entire documentation.

UPDATE
I'm still working on organizing files to make maintaining/contributing easy. I was able to reduce the amount of code in templates, but it took much longer than I thought :(

@jhoffner I found that Qualified has Knowledge Base so I decided to adopt the structure:

/languages for listing supported languages
/languages/:language for language overview
/languages/:language/:testing for test references

I also found some broken links while trying to reuse some of the contents. I left comments in relevant markdown so you can fix the Qualified docs when reviewing, but if you need to fix them sooner I can list the details here.

@jhoffner
I made the repository for reviewing and https://kazk.github.io/docs.codewars.com/.
I wrote some READMEs because commits are really messy and useless for reviewing.
There're some ideas that I'd like to know what you think, like more extensive use of YAML front matter and enabling to share documents with Qualified through variables and data files. I'll try to comment on those separately and point to related files so you don't have to dig through the mess.
For fixing Qualified docs, see markdown files containing "fixed" :)

I'm not finished with organizing but started to work on styling to maintain my drive.

@kazk this is looking really great!
I really like the structure you came up with. It looks clean and seems like it covers everything. How were you thinking of tying in detailed CLI documentation (like @config statements) and API documentation? Same site or linked?
I should have bandwidth next week to help work out some of the layout/design stuff. How easy is it to add a search bar to Jekyll?

How were you thinking of tying in detailed CLI documentation (like @config statements) and API documentation? Same site or linked?

I think setting up a GitHub Pages for the CLI repository and linking from this one is better for now.
From the visitor's point of view, it shouldn't make much difference if the pages are styled similarly and the navigations are bidirectional.
I'm concerned about redundancy though.  It's also hard to categorize some documentations.
For API docs, I haven't looked into integrating slate with Jekyll yet.
Is dev.codewars.com on GitHub Pages? I noticed that the changes are not reflected.
Is this related to #911?

I should have bandwidth next week to help work out some of the layout/design stuff.

:+1:

How easy is it to add a search bar to Jekyll?

Adding search shouldn't be hard:

Generate JSON representation of documentations using template
Use some client side JavaScript library making use of the data

This is somewhat related to my idea of using YAML front matter more extensively.
I wrote POC using YAML front matter extensively (unitkit-fm.md).
Currently, this produces equivalent result as unitkit.md, but has advantage of being much more flexible.
Pros:

Navigations like table of contents and sidebars can be generated using templates instead of relying on client side JavaScript.
Data used for searching can be improved.
Allows more complex markup, while letting contributors to continue using markdown.
Helps to keep consistency
Easy to change how informations are presented

Cons:

Browsing the file on GitHub :(

To workaround this I made a unitkit-fm2.html.

It doesn't show garbage and at least highlights HTML tags.
I also like that now I can link to a specific line: unitkit-fm2.html#L184.
What do you think of this approach?

By the way, Qualified's UnitKit documentation has 2 UKFloatsEqual and the second one should be UKFloatsNotEqual. The previous search with "fixed" didn't show this file because I used "corrected" :(

What do you think of this approach?

I think I like it. Pretty cool to use yaml like that within markdown. Is this a custom Jekyll thing for any type of file or just Markdown?
As long as the files don't become completely unapproachable because there is so much logic embedded in them, I'm good with this. Obviously KISS where we can so that more users in the community don't feel overwhelmed when trying to contribute - but it seems pretty clear to me. The other possible downside is that added complexity can result in things being more error prone, but it doesn't seem to be a huge risk.

It's Jekyll thing (some other static site generator also supports this) for any files.

The front matter is where Jekyll starts to get really cool. Any file that contains a YAML front matter block will be processed by Jekyll as a special file. The front matter must be the first thing in the file and must take the form of valid YAML set between triple-dashed lines.
https://jekyllrb.com/docs/frontmatter

As long as the files don't become completely unapproachable because there is so much logic embedded in them, I'm good with this.

I've been working to reduce amount of logic in templates for this reason.
I had some templates do site.docs | where: ... | group_by: ... and wasted huge amounts of time trying to figure out what went wrong :(
Now this is mostly handled by grouping them into smaller Jekyll collections then doing site.collection_name.
This approach simplifies generating data in similar way by utilizing Jekyll features instead of writing complicated templates.

Obviously KISS where we can so that more users in the community don't feel overwhelmed when trying to contribute - but it seems pretty clear to me.

Yeah, this was my concern. I'll make sure to keep this easy to contribute.

@jhoffner
Added simple search page example that just lists links to matched document.
The source is docs/_pages/search.html.

Uses lunr.js which supports some modifiers.

single term: foo returns documents matching foo.
multiple terms: foo bar returns documents matching foo OR bar.
documents matching both scores higher and are returned first.
wildcards: foo*
fields: title:javascript returns documents having 'javascript' in title.
This can be combined with other term modifiers.
title:foo* bar returns documents having foo* in "title" and bar in any field.
Unrecognized field will error.
boost: foo^10 bar treats foo more significant
fuzzy: allow n edit distances. foo~1 matches boo, but not boot

Index has fields title/content/url/collection.
Searches all collections.

Excellent search page, can't wait until the new Docs become official :heart:  One tiny thing though regarding the "PHPUnit Testing Framework" Test Reference - the correct order of arguments for TestCase::assertEquals() should be:
$this->assertEquals($expected, $actual[, $message]);

@DonaldKellett fixed :)

(reposting https://github.com/Codewars/codewars.com/issues/981#issuecomment-305420185)
@jhoffner
I think the workflow will be like:

Commit contents in kazk/docs.codewars.com to kazk/codewars.com:master for #900

echo docs.codewars.com > ./docs/CNAME

Merge #900
Setting > Select source: select "master branch /docs folder"
Setting > Custom domain: enter "docs.codewars.com"
Configure DNS

Enforcing HTTPS on custom domain might require further configuration

Publishing your GitHub Pages site from a /docs folder on your master branch

I'll start this in few days, hopefully tomorrow.
Before we start accepting contributions, we need to decide the license.
I usually make my code permissive (MIT/BSD-3-Clause/ISC/Apache-2.0), but I don't have strong opinions. Maybe aligning with codewars.com?

To promote collaboration and create a living resource, all kata solutions are actively opensourced under the FreeBSD 2-Clause License. All authored kata remain the IP of authors and contributors, with a license granted to Codewars, to maintain and advance the resource.
https://www.codewars.com/about/terms-of-service

I'm fine with whatever you choose.

@jhoffner
Note that this won't work properly on https://codewars.github.io/codewars.com because I've set the baseUrl in _config.yml to "" instead of codewars.com for custom domain.
See https://kazk.github.io/docs.codewars.com to get some ideas how it might look like.
This is far from complete, but I think it's good enough starting point.
Styling definitely needs some work and also need official assets.

That small svg logo is from my profile badge :sweat_smile:

Let me know if you have any questions.

Note that this won't work properly on https://codewars.github.io/codewars.com because I've set the baseUrl in _config.yml to "" instead of codewars.com for custom domain.

Apparently, it does work :/ https://kazk.github.io/codewars.com
I assumed baseUrl: project-name was required for project pages because I had read multiple issues related to this. Maybe Relative links for GitHub pages from 2016-12 is relevant.

@kazk for the license. I'm leaning towards MIT, though recently I discovered a site which is a complete rip off of Codewars. It utilizes the Coderunner CLI, content is clearly lifted (but slightly modified) and even the fake testimonials on the landing page are clearly just rewritten versions of our real testimonials. He is clearly trying to write off this "product" as his own work without giving any credit where credit is due.
Normally for documentation I would just think MIT but if someone is going to create a Codewars clone the last thing I need is them also ripping off our documentation. With that said... I'm probably still leaning towards MIT.
Note that I am not mentioning the actual website because I'm not going to give it free marketing.

He is clearly trying to write off this "product" as his own work without giving any credit where credit is due.

From Wikipedia, the exact wording of the license is as follows:
Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

@jhoffner Perhaps you should consider suing this imposter.  Has the imposter included this license anywhere in his/her counterfeit website?  If you could prove that he/she hasn't even bothered to include the license in the website then he/she has already violated this explicit condition in the license: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.  Apart from that, even if he/she included the license in his/her website, you could still sue him/her for fraud as long as you can objectively prove that the testimonials on his/her website are not genuine.

@jhoffner

recently I discovered a site which is a complete rip off of Codewars.

That's not cool :( Is there anything that you can do to stop?

I just googled and found "clonewars", the one with storm trooper. Is this it? It looks like a school project.

@jhoffner

Though the docs section of CW is terrible, so we need to also prioritize getting the new docs site ready. [...] I want to help get contribute some UI updates to the docs you built so that we can get that out ASAP.

I feel like I've put too much in this PR already considering the original goal to setup.
Can we work on improvements by opening separate PRs after merging this? Note that the site won't go "live" until you change the repository setting to enable GitHub Pages and point to /docs.
This way, we can open an issue for each requirements and I (or someone) can work on to fix that. This will also give me a chance to document trade-offs I've made while making this and some constraints that applies.
I tried my best to design the overall structure to be flexible and easy to maintain, but it's hard to do so without actual contents. I put together this PR as minimal viable after seeing high demands on new documentation and assumed that we're doing this progressively.
Things like menu and search are affected by this.
The top menu is minimal because I believe navigations needs to consider contents in order to be effective. I also valued consistency for now. There're no sidebars or table of contents for the same reason.
The search is not easily accessible because it's incomplete, just a PoC and needs feedback from testers.
Styling is not good because my CSS skill is low :( and HTML generated from Markdown is tricky to style. I was still experimenting, but I thought I should just ship it because people need updated documentation as soon as possible.

Can we work on improvements by opening separate PRs after merging this?

Yes absolutely.
BTW your initiative on this is much appreciated.  I've got it on my roadmap to dedicate some time to this starting the first sprint of July. Assuming things don't change, I can jump in then to help out with the UI and help coordinate things. I may be able to get around to some design work next week but it's hard to say.
In the meantime my mental buffer is a little overloaded and I haven't been able to fully realize everything that is needed for this project, so please just let me know what I can do to help and I will follow your direction. Eventually I will get caught up and will be able to help coordinate more.

please just let me know what I can do to help

I think the most important thing for now is license, other things can wait.
I'll open an issue asking for feedbacks and issues for areas that the community can help.

For the license, I'm between (cc-by-sa-4.0)[https://choosealicense.com/licenses/cc-by-sa-4.0/], which is the same license that Wikipedia uses and seems to be a good one for documentation, or FreeBSD 2-Clause, which is the same one that Codewars content is licensed under.
Anyone have any input on this?

Sometimes documentation and blogs have separate licenses for text and code.
Maybe CC-BY-SA-4.0 for text and BSD 2-Clause for code?

I think one of the reasons for having separate licenses is to allow people to use example code freely.
Can a user use a code snippet in the documentation on Codewars if we make the entire documentation CC-BY-SA-4.0? Codewars makes submitted content BSD 2 clause, but the CC-BY-SA requires CC-BY-SA or compatible license and BSD 2 clause is not one of them?
https://creativecommons.org/licenses/by-sa/4.0/

I'm not really sure how to best indicate dual-licensing, but that sounds like a plan to me.

I'm getting really confused which rules applies where and how :(
I don't know anything about legalities and above comments were from skimming the license.

I'm not really sure how to best indicate dual-licensing

The Go Blog has a line like the following for example

Except as noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code is licensed under a BSD license.

OK I will run this by Nathan (the business side of Qualified) tomorrow and see what he thinks. We may be able to get a lawyer to figure this out for us. We also might just be able similar text to the bottom of the documentation, similar to how the Go blog is. Having a license file in the repo isn't necessary required AFAIK, just a good practice. I think Github introduced the concept to make things easier.

OK it looks like simply including a blurb like the one on the go blog within the README is acceptable. So we could include this:
Except as noted, the content of this page is licensed under the CC-BY-SA-4.0 License, and code is licensed under a the BSD 2 clause.

I think the first one is better (and grammatically correct).  "Code" should automatically imply, well, all types of code which includes the code "examples" provided in the new Documentation.

Except as noted, the content of this site is licensed under the CC-BY-SA-4.0 License. Code,
including code examples, are licensed under the BSD 2 clause.

Possible additional phrase:
"Content" is considered any information contained within a markdown, or `.md` file.
All other files are considered "code".

Possible additional phrase:
"Content" is considered any information contained within a markdown (`.md`) file.
All other files are considered "code".

But wouldn't this imply that the code examples contained within the Markdown files are licensed as Site Content and not code?  Unless the Markdown files somehow "pull" the code examples from actual "code" files.


Fix Posts by Site API route param to use site_domain instead of site_url

PR to fix bug found in https://github.com/Charcoal-SE/metasmoke/issues/226#issuecomment-336497586 while reviewing API documentation for #226.

Thanks!


React-Based DMN Editor: use shared DMN models for testing purpose

DMN models used for testing purposes are moving to the kie-dmn-test-resources module.
Readme file gives a quick explanation of its structure.
It would be better to start using such files:

to be sure that both editor and engines works and behaves the same way
such files are tested/validated engine side, so we are sure about their validity

@tiagobento @yesamer

For team organization, this ticket will be prosecuted by @yesamer


Add suppress warnings in ca1707

This pull request is related to https://github.com/MicrosoftDocs/visualstudio-docs/issues/3971
Best practices for test naming convention is defined in https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-best-practices as:

The name of the method being tested.
The scenario under which it's being tested.
The expected behavior when the scenario is invoked.

And those three parts are represented in a method name separated by an underscore.
But the description's CA1707 warning is:

By convention, identifier names do not contain the underscore (_) character. The rule checks namespaces, types, members, and parameters.
When to suppress warnings
Do not suppress a warning from this rule.

I changed the suppress warnings with the suggestion.
Fixes #3971

@Leanwit : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@gewarren : Thanks for your contribution! The author(s) have been notified to review your proposed change.


[BUG] Monitoring application upgrade 5.5.36 -> 5.5.38 is failing with error

Describe the bug
---> Deleting old 'influxdb' resources
NAME       READY   UP-TO-DATE   AVAILABLE   AGE
influxdb   0/0     0            0           15d
error: error executing template "{{(index .items 0).spec.nodeName}}": template: output:1:3: executing "output" at <index .items 0>: error calling index: index out of range: 0

Next try of the job marks the upgrade as completed even when upgrade didn't happen:
[root@ip-172-31-0-5 ec2-user]# kubectl logs monitoring-app-update-8bbba8-zc8qk -n kube-system
---> Assuming changeset from the environment: monitoring-5511
---> Checking: monitoring-5511
no errors detected for monitoring-5511

Environment (please complete the following information):

OS [e.g. Redhat 7.4]:
Gravity [e.g. 5.5.4]: 5.5.38
Platform [e.g. Vmware, AWS]:

Duplicate of https://github.com/gravitational/gravity/issues/1281.


VMSS Instance upgrade does not retain data disk

Hi,
I have used Packer to build custom ubuntu linux image. I use terraform to create VMSS using data disks. When VM instance is created,  custom scripts are run and the data disk is attached. I am using cloud init script to initialise the disk. I can run df command and could see that data disk is attached.
When I create a new version of packer image and try to upgrade existing VM instance, the upgrade is done but the data disk attached to VM instance is lost. As VMSS uses managed disks, the details of the lost disk are not shown in the UI and I could not find any documentation on how to find the missing data disk.
Is it possible to retain data disk and attach to VM instance once upgrade is complete?

@gopikishanm please provide us with a link to the documentation you are following so we can better assist.

Hi,
I think it is issue with my disk mounting scripts. Once I upgrade linux custom image, the disk is attached but is not mounted. I had to update my custom script to mount the disk


Fix next / previous block buttons no longer showing on small screens

Because of my other PR (#145) being merged after the one that added the buttons on small screens (#148), it got overwritten and they no longer appeared (sorry @dated). I've re-added them, with some slight adjustments, so they show up again.
Since the "previous block" text takes up quite some space, I've changed the media query to 768px and the sm uses to md, otherwise it did not properly line up on screens that were between sm and md in size.

thanks for checking @ItsANameToo! i feared as much but had not the time to check ;) yea, the sm/md change is definitely necessary with the button text always shown! thanks for taking care of this.


fix: stop trying to release on main

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 49.584%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1014

Relevant Lines:
2045

💛  - Coveralls


converted 2 functions to templates

Converted extract_partial_bits() and extract_overlapping_bits() functions to templates.
This fixes linker duplication errors when the .hpp is included in 2 projects, because these functions had their definition in the hpp file.

Thanks for the patch. I'd like to ask a question before I apply it, though. I didn't think about the potential linker problem, but now that I'm aware of the issue, it seems to me that using an unnamed namespace is a natural way of dealing with it. Are templates superior in this case in your view?

Hi,
I'm not sure how unnamed namespace can help with this. Can you point me to an example?
The problem I had is including basen.hpp in a static library project and then including basen.hpp in a project that links with that library. 2 methods where duplicated according to the linker error, as indeed their body is included in each project. Even if the functions are inside a namespace, there  is still 2 copies of them, just with name like bn::converted extract_partial_bits()
In case of a template, actual function is not generated util the template is used.

I've created an example with basen.hpp itself. Please try 81eb445fb8135d8f9a14810eef1f53cf43f1e8e0

I've tried your solution and it works perfectly. Thanks for the fix!
I'm just closing my pull request

Great to hear that it works for you. Thanks for testing!


Convert boolean parameter values to strings

Addresses part of #102

So all parameters should be strings?

Any reason not to merge this?  Seems simple / straightforward.

Thanks!


Bug fix. Do not push to incoming_queue after it is flushed.

LGTM. Thanks for this. Will merge when green.

Travis green~


ACM-4246 immediately show hcp upgrade

backport of https://github.com/stolostron/console/pull/2707

/retest

/cc @fxiang1


[PTRN-1] Improve clog dependency

In this commit, they actually remove the external dependency to clog. Your work is essentially reverting that. https://github.com/Maratyszcza/cpuinfo/commit/3dc310302210c1891ffcfb12ae67b11a3ad3a150


Fix division by zero in lateral_planner.py

lateral_planner.py:191: RuntimeWarning: invalid value encountered in double_scalars
When v_ego = 0 and OP activated we are getting division by 0 which sets value to nan, so param angleSteers becomes set to nan and could potentially break other code.
How to reproduce: TSS2 Corolla, shift D, enable brake hold, enable ACC, enable OpenPilot. tmux will throw an error and ZMQ will publish lateralPlan with param angleSteers: nan while car is not moving.

Or != 0 would be better? Sometimes we are getting negative v_ego

Hmmm, I don't really like that logic. Since if v_ego is super small it can potentially output weird things.
I think the best is to replace v_ego with max(v_ego, 1e-1). Can you do that?

Done

Think I need to add a new reference


Edit menu items crash electron for windows os #2571

electron doesn't crashes for windows and unix but i can't test it on mac os. could you please do this?
select all menu item doesn't work yet

:+1:


Issue/5842-dont-run-cleanup-jobs-halted-environment

Description
The server runs many cleanup jobs in the background. This issue should make sure that no cleanup jobs are run on halted environments.
Closes #5842
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[x] Attached issue to pull request
[x] Changelog entry
[x] Type annotations are present
[x] Code is clear and sufficiently documented
[x] No (preventable) type errors (check using make mypy or make mypy-diff)
[x] Sufficient test cases (reproduces the bug/tests the requested feature)
[x] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )
[ ] If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see test-fixes for more info)

@arnaudsjs EnvironmentMetricsService._cleanup_old_metrics() was already done, I've added a test

Processing this pull request

Failed to merge changes into iso5 due to merge conflict. Please open a pull request for these branches separately by cherry-picking the commit that was made on the branch master (git cherry-pick cc846dcc71f81deb85266ff1ee5402009d928ad8).

Merged into branches master in cc846dcc71f81deb85266ff1ee5402009d928ad8

Not closing this pull request due to previously commented issues for some of the destination branches. Please open a separate pull request for those branches by cherry-picking the relevant commit. You can safely close this pull request and delete the source branch.

This branch was not deleted as it seems to still be in use.

Processing #5915.


[README] Create a meaningful description of the problem that this package aims to solve

There's will be great to have some examples of the encountered problems

Hi @kitsuniru,
I’ll release the first version within the next week or two.
The concept is similar to the wolt_modal_sheet’s resizable nested-navigation dialog. It’s a great package, but it has a limitation: it doesn’t support official navigation APIs such as Navigator.push/pop and Navigator 2.0 (go_router, auto_route, etc.).
The resizable_navigator package provides the very basic components necessary to create such a self-resizing Navigator widget.


Set the standard-indent variable

This is used in a lot of places to specify standard indentation.  Web-mode, for one, uses it to determine the standard indentation.

looks fine


problem with set_mask_feature

Adding a land mask with,
s1meta = xsar.Sentinel1Meta(safe_path)
s1meta.set_mask_feature('land', '/home/datawork-cersat-public/cache/public/ftp/project/sarwing/xsardata/land-polygons-split-4326/land_polygons.shp')
dataset_1000m = xsar_obj_1000m.datatree['measurement'].to_dataset()

dataset_1000m["land_mask"].values

leads to this error :
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[28], line 1
----> 1 dataset_1000m["land_mask"].values

File /home1/datawork/vlheureu/conda-env/testEnvXsar/lib/python3.10/site-packages/xarray/core/common.py:276, in AttrAccessMixin.__getattr__(self, name)
    274         with suppress(KeyError):
    275             return source[name]
--> 276 raise AttributeError(
    277     f"{type(self).__name__!r} object has no attribute {name!r}"
    278 )

AttributeError: 'DataArray' object has no attribute 'values'

instead of returning an array.
(testing safe was
safe_path = "/home/datawork-cersat-public/cache/project/mpc-sentinel1/data/esa/sentinel-1a/L1/EW/S1A_EW_GRDM_1S/2018/282/S1A_EW_GRDM_1SDV_20181009T234410_20181009T234510_024066_02A153_DB9C.SAFE"

)

re open this ticket since, with last PR #159  we have now some acquisition with incorrect (only ocean) land_flag content:

RS2_OK135107_PK1187782_DK1151894_SCWA_20220407_182127_VV_VH_SGF


Add test for KeyValueColumnize

In relation to #1214, this PR just adds a test to demonstrate the way KeyValueColumnize deals with records and therefore requires to see the rows in a particular order to work correctly.
The point is just to make sure that the existing functionality is preserved by future contributors. And increase our test coverage, which seems to be very low at the moment.

Whoops, curious that there is this discrepancy between OpenJDK and OracleJDK. Thank you Travis!

@thadguidry that's a good idea! feel free to add that on the branch (it's a branch in this repo so you can push to it)

@wetneb liking Codacy so far ?

@thadguidry yeah it's clearly better than coverall with the code review features!

@wetneb OK.  For this one, as long as we get the additional test and things pass and Travis is happy and no errors (warnings OK for now) on Codacy, then you can accept commit.  We'll probably want to tweak Codacy for our project eventually (ignoring a few things, etc)... we should probably schedule a hangout with Jacky after summer break..September-ish.  Sound good ?

@thadguidry yeah it'd be good to configure it a bit so that only relevant issues are reported. Sure, happy to hangout when you want!

The code review features are nice but I think the most important thing is the code coverage. Because 20% coverage is really low for a big project like that.


Prevent multiple Actions with identical routing annotations

This causes a server start up failure, which should result in catching
the error in a unit test before committing the bug.
2 actions with identical routing annotations is most likely a bug
introduced by bad copy/paste. One of the Actions will never be
dispatched, so it provides no value.

Thanks @ryanhall07 !


changed border radius to be round by default

Added logic for the default borderRadius value to be half the width (of course only if provided).

It would also be nice if you can make corresponding change to the readme section about the default value of indStyle.

I've made the changes that you've asked. Is the same logic for horizontal scroll relevant as well?

Sorry about the late response, but the change looks good and your code has been merged.


Exporting as Excel: Date fields come out in the format dd-mm-yy

Is it possible to change the default date being outputted to yyyy-mm-dd?
The dates are still working on a technical level in Excel (stored properly and can use date formulas on them), but the formatting being applied is currently dd-mm-yy.

Im seeing on the duckdb page the following:

Dates and timestamps are currently not supported by the xlsx writer. Cast columns of those types to VARCHAR prior to creating the xlsx file.

I can cast them to varchar, and format them accordingly.

If they are VARCHAR though Excel might not recognize it as a date column anymore which loses a lot of functionality. Would have to test it

To answer the question, i dont think there's functionality on the duckdb API for datetime formatting.
I find it curious that currently the export contains dates that work on a technical level, while the documentation says otherwise. Date and timestamp columns shouldnt even be supported.

To answer the question, i dont think there's functionality on the duckdb API for datetime formatting.
I find it curious that currently the export contains dates that work on a technical level, while the documentation says otherwise. Date and timestamp columns shouldnt even be supported.

I only tested it with date and it works where I can filter the column in excel down to year/month and apply date functions to it immediately. I am not sure about datetime.


Misc entity fixes

Resolve #236
Resolve #237
Resolve #238
Resolve #239
Resolve #240
Resolve #241
Resolve #242
Resolve #243
Resolve #244
Resolve #245
Resolve #246
Resolve #247

Alr should be done now


GUI Gtk

I could do a graphical interface in Gtk 3, would you agree?

Hi, thanks for your interest in this project. I designed this tool to be used mainly through a terminal, but if you can provide a nice looking graphical interface I'll consider it for sure. I never used Gtk before, so this could also be an opportunity for me to learn something new. Thanks!

Ok, great hands to the work, one more question, it is possible to return the url of the application icon. Sorry for my English

You should be able to get the icon's url with some code similar to this:
api = Playstore(credentials)
app = api.app_details(package_name).docV2
for image in app.image:
    if image.imageType == 4:
        response = requests.get(image.imageUrl)
        if response.status_code == 200:
            # Save the icon into a file or do whatever you want with it.
            with open('icon.png', 'wb') as f:
                f.write(response.content)
            break

I begin to work in https://github.com/JimScope/apkdown is still in alpha and don't download yet mi internet is a shit, contact me<EMAIL_ADDRESS>when you read this

I begin to work in the GTK+ version. its work but only in retrieve app details, don't download yet because is very dificult to me testing without internet, maybe tomorrow works the download part. Its buggy and is incomplete so don't hurt me. the way for contact me is for email<EMAIL_ADDRESS>When you read this, write me for explain the issues with mi jobs environment.

I begin to work in the GTK+ version. its work but only in retrieve app details, don't download yet because is very dificult to me testing without internet, maybe tomorrow works the download part. Its buggy and is incomplete so don't hurt me. the way for contact me is for email<EMAIL_ADDRESS>When you read this, write me for explain the issues with mi jobs environment.


Style Guide - Break down routes based on levels

Originated from a conversation with @delasteve about how grouping should be handled in the routes file.
Why

Developers don't want to dig through multiple routes files to see where the routes end
Routes with no children should live in their parent route

const ROUTES = [
  {
    path: '',
    component: HomeComponent
  },
  ...PARENT_ROUTES
]

const PARENT_ROUTES = [
  {
    path: '/parent',
    component: ParentComponent,
    children: [
      ...CHILD_ROUTES
    ]
  }
]

const CHILD_ROUTES = [
  {
    path: '/child',
    component: ChildComponent,
    children: [
      {
        path: '/grandchild',
        component: GrandChildComponent
      }
    ]
  }
]

👍
There are two extremes, one where all routes are defined at the top level, and the other where they are using the spread operator for child routes. I think this is a good compromise. @brandonroberts nails it with the first bullet point as to why this compromise works.

Closing as the convention has been resolved.


Add Meteorology Extension

This is my first extension and its based around meteorology as it interests me. It allows you to communicate with the weather.gov api, convert between temperature units, and have access to geolocation permissions. I would like feedback for changes if possible.

It doesn't work for the desktop version, the request geolocation block doesn't do anything and the extension freaks out as a result

If you already accepted geolocation it wont ask again hence why it may not work, what browser are you on?

the desktop version
No browser at all, maybe that is an issue with the Turbowarp App but other extensions with similar functionalities work just fine

Oh then that is an issue with it, geolocation can only work on a chromium browser. Don't really know how to fix that.

Oh then that is an issue with it, geolocation can only work on a chromium browser. Don't really know how to fix that.

Weird. The desktop version is based on Electron, that is based on Chromium. I don't think Electron apps can request geolocation, as stated in here.

Geolocation support in Electron requires the use of Google Cloud Platform's geolocation webservice. [...], place the following code in your main process file, before opening any browser windows that will make geolocation requests.

So your extension is good and the issue can't be fixed.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-source-version-target/pull/214 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-source-version-target.git

Docs Build status updates of commit 16265c9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

versioningArchiveSourceFolder/TOC.yml
:white_check_mark:Succeeded
View (sql-server-2014 || azuresqldb-current || azuresqldb-mi-current || azure-sqldw-latest || >= aps-pdw-2016 || >= sql-server-2016 || >= sql-server-linux-2017 || sql-server-previousversions || sqlallproducts-allversions)

versioningArchiveSourceFolder/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Figures from paper are NOT reproduced

The figures for example 01 and example 02 are not the same as those within the Quast & Wagner paper. Seems that there is still something wrong here.
@raphaelquast can you please check if you can reproduce this error?
Examples can be found in the documentation branch.

@bulli92 There are some issues that can be solved and some that will remain...

changed plot to display intensity instead of sigma_0  (by setting sigma0 = False)
changed normalization of BRDF to fit the definition in the paper (by setting NormBRDF = np.pi)

unfortunately as I already mentioned there was an error in the definition of the
program I used to generate the plots for the paper... (I will try to get this corrected)
The error originated from the fact that the arguments of the functions (as also implemented
in our code) are in terms of zenith-angles... the arguments in the theory-part of the paper are in
terms of a spherical coordinate-system!
-> this resulted in an error in the definition of the fn-coefficients and consequently in an
error in the calculation of the interaction-term.  (unfortunately the arguments of the functions
have been similarly defined for numerical testing which then resulted in an agreement of the results)
In example 1 this has no effect on the backscattered radiation due to the symmetry-properties
of the Rayleigh-phase-function. In example 2, the interaction-term is now much more significant.
This is reasonable since the BRDF is defined to scatter primarily in specular direction
(away from the detector) and the phase-function scatters primarily in forward-direction
(also away from the detector). So for t_0 > 40 deg most radiation that is
scattered in direction of the detector originates from multiple-scattering contributions.
Also the fact that the interaction-term in example_2 at t_0 = 0   is much larger than in example_1 is reasonable since the phase-function now scatters primarily in forward-direction (and the BRDF is the same).

o.k., then it's perhaps in particular a matter how to DOCUMENT these
differences.
A user will compare the example figures with those of the paper, like I
did, and wonder why things look different. Thus we need to highlight
that somehow.
On 3/28/17 9:54 AM, raphaelquast wrote:

@bulli92 https://github.com/bulli92 There are some issues that can
be solved and some that will remain...

changed plot to display intensity instead of sigma_0 (by setting
sigma0 = False)
changed normalization of BRDF to fit the definition in the paper
(by setting NormBRDF = np.pi)

unfortunately as I already mentioned there was an error in the
definition of the
program I used to generate the plots for the paper... (I will try to
get this corrected)
The error originated from the fact that the arguments of the functions
(as also implemented
in our code) are in terms of zenith-angles... the arguments in the
theory-part of the paper are in
terms of a spherical coordinate-system!
-> this resulted in an error in the definition of the fn-coefficients
and consequently in an
error in the calculation of the interaction-term. (unfortunately the
arguments of the functions
have been similarly defined for numerical testing which then resulted
in an agreement of the results)
In example 1 this has no effect on the backscattered radiation due to
the symmetry-properties
of the Rayleigh-phase-function. In example 2, the interaction-term is
now much more significant.
This is reasonable since the BRDF is defined to scatter primarily in
specular direction
(away from the detector) and the phase-function scatters primarily in
forward-direction
(also away from the detector). So for t_0 > 40 deg most radiation that is
scattered in direction of the detector originates from
multiple-scattering contributions.
Also the fact that the interaction-term in example_2 at t_0 = 0 is
much larger than in example_1 is reasonable since the phase-function
now scatters primarily in forward-direction (and the BRDF is the same).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pygeo/rt1/issues/35#issuecomment-289692188, or
mute the thread
https://github.com/notifications/unsubscribe-auth/ACt0C3yCGyn8HmwmZDBUag369nQ2hRreks5rqLzDgaJpZM4MrLNy.

yes I agree...
I planned on contacting OSA once our module has reached a certain degree of
completion so that i can directly refer to it.
One way of documenting it in the meanwhile would be to just show how to reproduce
the error of the paper.... This can be done easily by changing the generalized scattering
angle parameter before evaluating the interaction-term and switching it back once the
coefficients have been stored, i.e.:
V.a = [1.,1.,1.]
R = RT1(I0, 0., 0., 0., 0., RV=V, SRF=SRF, geometry='mono')
fn = R.fn  # store coefficients for faster iteration
V.a = [-1.,1.,1.]

This change is equivalent to setting  p(t_0, t_ex, p_0, p_ex )        --->         p(np.pi - t_0, t_ex, p_0, p_ex )  during the evaluation of the fn-coefficients which is exactly the error that occured within the paper-plots
... to see the equality of the transformations:
for a[0] = a,   a[1] = a[2] =1. we have:

cos(THETA) =   a * cos(t_0) cos(t_ex)         + sin(t_0) sin(t_ex) cos(p_0 - p_ex)
           = - a * cos(np.pi - t_0) cos(t_ex) + sin(t_0) sin(t_ex) cos(p_0 - p_ex)

o.k., where is the right place to but these additional details?

I think it's most reasonable to add it to the end of the IPython notebook files
(as a code that one can un-comment to see the changes) and maybe a short note in the examples-section
that refers to the IPython files.
I'll try to add it today!
(I will not add a lot of explanation since it would mainly be an explanation of how to do it wrong...)

I added an explanation to the ipython notebook of example 2.
From my side I think that's enough for the time being ...
If you're OK with it we can close this issue


Nested objects

Nested objects appear primarily in two places:

Within parameters, such as an enum for a query
Within subobjects of a larger object

It is proposed that these are handled as follows:

Generate an 'anonymous' name for the model from the field name that specified it.
Add that name to a list of extra models to be generated, and check for key uniqueness.
Deep equality check for uniqueness, and:

If found to be equal to another key, just use that one
If not equal, generate a unique name for the key in some way

Add them to the models to be generated

Fixed!


[Bug]: Check extra741, 742 throws detect_secret error; sometimes fails for same resource sometimes passes

What happened?
Check Extra 7.41 and 7.42 show status as pass and fail for the same resource on different occasions. It also throws a detect_secret syntax error while having the latest version or detect_secrets library installed.
How to reproduce it
Steps to reproduce the behavior:

sudo ./prowler -c extra741

sudo ./prowler -c extra742

Single account environment

Detect_secrets error:
Traceback (most recent call last):
File "/bin/detect-secrets", line 5, in
from detect_secrets.main import main
File "/usr/lib/python2.7/site-packages/detect_secrets/init.py", line 1, in
from .core.secrets_collection import SecretsCollection  # noqa: F401
File "/usr/lib/python2.7/site-packages/detect_secrets/core/secrets_collection.py", line 18
path: str
^
SyntaxError: invalid syntax

With an earlier version of detect_secrets the same resource shows as PASS which now shows as FAIL

Expected behavior
Per the above syntax error the check execution stops if -M command is provided with a format to be saved. The error stays the same with latest version of prowler tool. Current version in use:2.4.0-07042021.
Screenshots or Logs

From where are you running Prowler?
Please, complete the following information:

Resource: EC2 instance
OS: Amazon Linux 2
AWS-CLI Version [aws --version]:aws-cli/1.18.147 Python/2.7.18 Linux/4.14.232-177.418.amzn2.x86_64 botocore/1.18.6
Prowler Version [./prowler -V]:2.4.0-07042021
Shell and version:GNU bash, version 4.2.46(2)-release (x86_64-koji-linux-gnu)
Others:detect-secrets-1.1.0

Additional context
Could you please look into this.

Hi @romgitrom we are investigating it but last version of detect-secrets is v.1.2.0 https://github.com/Yelp/detect-secrets/releases/tag/v1.2.0.
Also, please re-test this issue with Prowler v.2.8.0, which is the latest release https://github.com/prowler-cloud/prowler/releases/tag/2.8.0
Also, we've identified a bug in check742.

Please, @romgitrom check this PR https://github.com/prowler-cloud/prowler/pull/1078 and test it with the latest version of  detect-secrets.
Thank you!!

Hi,
Thanks for looking into this, after updating prowler to v2.8.0 and detect_secrets to 1.2.0 there were some different errors.
python --version
Python 3.7.10
with detect_secrets==1.2.0:-
pip3 install detect_secrets==1.2.0

sudo pip3 install detect_secrets==1.2.0

sudo ./prowler -c extra741

with detect_secrets==1.1.0:-

Could you please help me understand if it is a python version or detect_secrets or prowler tool issue.
Thank you in advance.

Hi @romgitrom, I think you have some kind of problem with your python environment.
I can't reproduce your issue. Please follow these steps to test another approach:

Run virtualenv -p $(which python3) venv && source venv/bin/activate
Run pip install detect-secrets

As you can see is a fresh new environment.
Then, ./prowler -c extra741 -f us-east-1 (I have included -f to check only one region)

Hi @romgitrom, we are closing this issue. Please, feel free to reopen it if you can provide us more information.
Thanks!


Feature request on refstats

Hi,
I've been fiddling with the refstats and it's really great to have the options but I was wondering if it was possible to do some more basic calculations such as mismatch rate and q-score across coordinates?
Thanks!
Chang

I think that should be possible! Just to clarify, are you talking about basecalled alignment statistics, like basecalled read/reference mismatches? I think the easiest way to implement that might be to output the read k-mer, q value, and maybe indel base count? A single base substitution could be inferred by comparing read/reference k-mers, and indels would be represented in a separate column as +/- values. I had something like that implemented in an early development version, but may take some time to re-implement for the current BAM format.


Shrinking class overhaul

Restructured the project by removing content from shrinking and moving it to other classes
fixes #32
fixed vents not sussifying for host (and sometimes client too)

MY MAN


Adding cardinality support for Java Functions

This PR includes cardinality support for following the standard of EventHub.
https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs-trigger?tabs=javascript#configuration
Overview
The cardinality is supported on the Functions Host side. If you set the Cardinality as Many, you need to set dataType as well. It is used for de-serialize the Kafka event to be the type of KafkaTrigger function. We can choose string, binary, stream as a dataType. I also consider if I should create DataType enum.  I follow the solution of the EventHub. They use string as the  dataType.
Includes

Cardinality support on Annotation
Cardinality and Confluence support sample
Document update and small bug fix
Cardinality sample is tested and run with Confluent cloud

Next step
For the Java users, I'd like to write Getting Started and reference documentation. For the GA, it might help the customers since building the Kafka development environment is slightly complex.
Additional Note
The documentation doesn't mention dataType.  I create an issue for it.
https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs-trigger?tabs=javascript#configuration
Currently, E2E of cardinality Many is commented.
https://github.com/Azure/azure-functions-java-worker/blob/dev/endtoendtests/Azure.Functions.Java.Tests.E2E/Azure.Functions.Java.Tests.E2E/EventHubsEndToEndTests.cs#L134

Thank you so much @TsuyoshiUshio .
Did we test that when we set the cardinality value it will pickt he right FunctionExecuter,
Cardinality one: SingleItemFunctionExecutor
Cardinality Many: MultipleItemFunctionExecutor

@ahmelsayed I tested.  When I set it as Cardinality.One, the function interface should be String, when I set Cardinaliry.Many, then the function interface should be String[] . I tested both cases. It works as I expected. And cardinality works perfectly. Cardinality.Many enables batch execution, when I post several message at the same time.

Thank you so much @TsuyoshiUshio .
That is great I just have more questions,
First, did you test the Kafka extension will pick up the right FunctionExecuter? My basic understanding is we support send data in batches using MultipleItemFunctionExecutor, adding @fbeltrao to correct me.
Cardinality one: SingleItemFunctionExecutor
Cardinality Many: MultipleItemFunctionExecutor
Second, The string[], did you get multiple items or just one?
Third, what will happen if we have Cardinality.One with []String?
Thank you so much,
Ahmed

Hi @amamounelsayed
I'd like to answer the question.
First
I think you are right. :) the Single or MultipleItemFunctionExecutor is decided by singleDispatch  that is coming from Azure Functions Host side.  In case of the parameter type is array, it is become false. The singleDispatch is created in here. https://github.com/Azure/azure-webjobs-sdk/blob/b798412ad74ba97cf2d85487ae8479f277bdd85c/src/Microsoft.Azure.WebJobs.Host/Bindings/BindingFactoryHelpers.cs#L56
The parameter is created here. https://github.com/Azure/azure-functions-host/blob/d87c8b0a972800434455b74e8253bbf2675956b9/src/WebJobs.Script/Binding/GeneralScriptBindingProvider.cs#L62-L101 depends on the Cardinality, the parameter will be array or non-array on this code.
Second
We can use String[] only we specify Cardinality.Many. In this case, it can receive several messages at a time.
Third
I've never tried. I'll try it and share it soon. :)

Hi @amamounelsayed
I confirmed that the feature works properly.
host.json
{
  "version": "2.0",
    "logging": {
      "fileLoggingMode": "always",
      "logLevel": {
        "default": "Information",
        "Host": "Debug",
        "Function": "Debug",
        "Host.Aggregator": "Debug"
      }
    }
}

Then, I can see the kafka extension's log.
on $TMPDIR//LogFiles/Application/Functions/Host I found the host log.
Cardinality.MANY
Function
@FunctionName("KafkaTrigger-Java-Many")
public void runMany(
  @KafkaTrigger(
    topic = "message",
    brokerList="%BrokerList%",
    consumerGroup="$Default",
    username = "%UserName%",
    password = "%Password%",
    authenticationMode = BrokerAuthenticationMode.PLAIN,
    protocol = BrokerProtocol.SASLSSL,
    cardinality = Cardinality.MANY,
    dataType = "string"
  ) String[] kafkaEventData,
  final ExecutionContext context) {
    for (String message: kafkaEventData) {
      context.getLogger().info(message);
   }
}

Log
2020-06-09T05:28:09.929 [Information] ReaderAsync called: MultipleItemFunctionExecutor

Cardinality.ONE
Function
    @FunctionName("KafkaTrigger-Java-One")
    public void runOne(
            @KafkaTrigger(
                topic = "message",
                brokerList="%BrokerList%",
                consumerGroup="$Default",
                username = "%UserName%",
                password = "%Password%",
                authenticationMode = BrokerAuthenticationMode.PLAIN,
                protocol = BrokerProtocol.SASLSSL,
                cardinality = Cardinality.ONE,
                dataType = "string"
             ) String kafkaEventData,
            final ExecutionContext context) {

                context.getLogger().info(kafkaEventData);

    }

Log
2020-06-09T05:36:28.893 [Information] ReaderAsync called: SingleItemFunctionExecutor

Conclusion
According to the Cardinality, the FunctionExecutor is switched as we expected.  See
https://github.com/Azure/azure-functions-kafka-extension/pull/137


[Analysis]  Filter analysis fixes

[x] Show 3rd form only when selecting a column above.
[x] 3rd should have "show" activated by default
[ ] Update description on analysis modal.
[ ] Is greater than / Is less than input label should be "value" in both cases.
[ ] Input text for string values should show a list when having less than XXX different values.

@saleiva: I couldn't find a valid description for this analysis in the designs (there we just have Transform your data into coordinates and also Calculate area o influence of a layer determined by walking or driving time or just by distance), so what about something like: "Filter the data based on the data of a column"

Sounds good,
"Filter your rows depending on the values of a column" ?

Great!