Fix 'permisison denied' on /var/logsupervisor

When does this issue occur? Seems strange that the container should be run in privileged mode.

is issue happen when build the phpdockerized image
here is the log :
front_1  | /usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
front_1  |   'Supervisord is running as root and it is searching '
front_1  | Traceback (most recent call last):
front_1  |   File "/usr/bin/supervisord", line 9, in <module>
front_1  |     load_entry_point('supervisor==3.0', 'console_scripts', 'supervisord')()
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 360, in main
front_1  |     go(options)
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 370, in go
front_1  |     d.main()
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 77, in main
front_1  |     info_messages)
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/options.py", line 1317, in make_logger
front_1  |     stdout = self.nodaemon,
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/loggers.py", line 344, in getLogger
front_1  |     handlers.append(RotatingFileHandler(filename,'a',maxbytes,backups))
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/loggers.py", line 172, in __init__
front_1  |     FileHandler.__init__(self, filename, mode)
front_1  |   File "/usr/lib/python2.7/dist-packages/supervisor/loggers.py", line 98, in __init__
front_1  |     self.stream = open(filename, mode)
front_1  | IOError: [Errno 13] Permission denied: '/var/log/supervisor/supervisord.log'

My machine:

Fedora 24
Docker version 1.10.3, build 19b5791/1.10.3
docker-compose version 1.7.1, build 6c29830

but the wird thing is it works fine on my windows machine, i think the problem is on my docker installation or the docker version that im using on my fedora, so should i close this pull request?

Yeah, that sounds more like an environment issue. Thanks though!


rename configMissingOK field to baseConfigMissingOK

What does this PR do?

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

Current dependencies on/for this PR:

main

PR #14833

PR #14854

PR #14855   👈

PR #14814

This comment was auto-generated by Graphite.

merged into https://github.com/DataDog/datadog-agent/pull/14884


Naoe index design

I tried making the checklist look more important to make sure that everyone goes through it.
https://user-images.githubusercontent.com/56588774/116611316-88009600-a936-11eb-8255-b358bbda2295.mov
Also, I changed "Miniconda or Anaconda" to emphasize recommendation of Miniconda rather than Anaconda.
I would like to implement reason why CR recommends Anaconda rather than Miniconda (ref. Zulip "installation instructions" thread in lessons stream, so I keep this as draft pull request.

I expected that this comment previews the mov file, but it did not. 😢


paired-end fastq files

Hi,
Thank you for the great tools.
I am wondering if I can get a  paired-end alignment using paired read files (e.g., *_R1.fastq.gz & *_R2.fastq.gz  )?
Thank you

Dear @farmohh,
Our software currently doesn't support paired-end reads, we will take this into account for future updates.
Sorry for any inconvenience.
Best,
Cristina


How remove the Feature.REGEX_BACKTRACKING_LIMIT

Hello,
I would very much like to use the lunr solution.
But I get the following error message:
[INFO] Building project .github/dita-ot/html.xml
     [xslt] Regex backtracking limit exceeded processing ^[....]*([....]+?)[....]*$\Z. Simplify the regular expression, or set Feature.REGEX_BACKTRACKING_LIMIT to -1 to remove this limit.
     [xslt] Failed to transform document: Failed to transform document: Regex backtracking limit exceeded processing ^[....]*([....]+?)[....]*$\Z. Simplify the regular expression, or set Feature.REGEX_BACKTRACKING_LIMIT to -1 to remove this limit.

An index is also not built see:
https://github.com/tboenig/t_guidelines/blob/gh-pages/html/js/lunr-client.js
Here my project/repo:
https://github.com/tboenig/t_guidelines
I have already tried several things.

I assume there is a quantity problem (regex..)
Somewhere a parameter is not set correctly.

Thank you.

Danke Matthias, glad to hear you found this and are interested in using it.
We haven't published an official release yet as we're still polishing a few rough edges, but user feedback is the best way to improve something like this, so thanks for taking the time to report this issue.
I haven't seen this error so far, so I'm wondering if there's something peculiar to your project that triggers this. 🤔

Thank you very much for the feedback.
I was looking for specifics. In the process, I may have found one:
<chapter href="trans/structur_gt.dita">
        <topicref href="structur_gt.ditamap" format="ditamap"/>
</chapter>

I add another ditamap to the ditamap. To the best of my understanding, this is DITA conforming.  I have now removed this feature.
The warning "...Feature.REGEX_BACKTRACKING_LIMIT to -1..." is now no longer reported. But my chapter is missing.  :slightly_frowning_face:
The search index is still not build.
When using: <publication transtype="html5">
Also does not abort processing a ditamap with a ditamap recursion.
Thanks again.

It is always good to get another test case document set. The problematic regex in this case was the following line:
       <xsl:element name="text">
-        <xsl:value-of  select="replace(replace($TEXT , '^\s*(.+?)\s*$', '$1'), '^ .*$', '')" />
+        <xsl:value-of select="normalize-space($TEXT)"/>
       </xsl:element>

There were a couple of extra issues regarding stemming and stop words, but I've included a fix for those as well.

If you want to run the patch before it lands on develop, amend your action as follows:
        plugins: |
          https://github.com/jason-fox/fox.jason.extend.css/archive/master.zip
          https://github.com/infotexture/dita-bootstrap/archive/master.zip
+         https://github.com/jason-fox/dita-bootstrap.lunr/archive/feature/offline.zip
          https://github.com/jason-fox/fox.jason.prismjs/archive/master.zip
          fox.jason.favicon

You'll need https://github.com/infotexture/dita-bootstrap/archive/develop.zip then.

puh,
the action workflow is running again.

unfortunately, an index is not formed.
-- isn't my dita project too big?
but the localization works now

If I manually copy over preview.json and search_index.json - it works properly https://jason-fox.github.io/t_guidelines/html/trans/ - this would seem to  indicate a fault in the GitHub Action.

I think that the underlying issue may be  reduced to a combination of running the dita-ot/dita-ot-action and creating a document in not in English For German to work, I think that the DE-DE locale needs to be  generated within the action:
locale-gen de_DE.UTF-8
LANG="de_DE.UTF-8"
LANGUAGE="de_DE:de"
LC_ALL="de_DE.UTF-8"

dita -f html5-bootstrap -i ...etc

The following GitHub Action is building your index successfully in German: https://github.com/jason-fox/t_guidelines/blob/main/.github/workflows/dita.yml -it is using the version found in PR #10 10


Marathon-LB enhancements.

Respect health checks by default
Supply some better default templates
Implement workaround for Alpine Linux DNS issue (it does not respect
ordering of DNS servers, and we include a non-Mesos-DNS entry on
DCOS)

cc @discordianfish

Looks good in general, although we need to find a different workaround for the musl issue.
Either we convert this to debian or we create our own base image. In that case I would fork https://github.com/sdurrheimer/alpine-glibc which is what the prometheus team is using as well and so far we didn't have problems wit that.
Regarding the templates: If we feel like those are more reasonable defaults, I would suggest to change the defaults in the script.

I split out the DNS change.

On another note: what should we do about reserving ports? We should probably include port 9090 in the marathon.json, and perhaps also a range of ports to which we might attach services.

@brndnmtthws Reserving ports is definitely an issue. Right now marathon-lb binds to the 'service ports' and it can't know them when scheduled, so it's very prone to port conflicts. We can disable that and have it only listen on 80/443 for incoming requests but then people can't use the service ports. So that's a product decision.

Could we be a little more prescriptive about port management? At the very least, we could make a recommendation (i.e., start numbering your service ports from port 10000 or something).

@brndnmtthws Yes, we could do that. Allocate 10000-20000 in marathon-lb and advise people to only use service ports in that range.

Unfortunately Marathon doesn't let you reserve a range of ports, so we'd have to iterate 10k ports in the JSON. Maybe we'll go with 10000-10100, and if someone needs more ports they can add them :)

Oh.. Hoped that would be possible. Well in that case I would probably just ignore this for now - or disable the service ports by default. To properly solve this we need a framework dynamically allocating the service ports on the public slaves.

Updated the PR. Removed the templates, and instead set those as defaults.

@brndnmtthws @discordianfish any chance to see this merged this week?

@brndnmtthws ^- Can you look into this?

@scalp42 Interesting. I don't see how the changes in this PR might affect that. I did notice, however, that the logs from the 2 examples above are slightly different. Namely, the first one attempts to reload HAProxy, whereas the second does not (servicerouter: reloading using /etc/init.d/haproxy reload...)

Right @brndnmtthws I didn't paste the full log, the reloading is working fine in both cases.
It's just the iteration of apps that doesn't work anymore between master and your branch (which I would need for the health checks from Haproxy).
I'm hoping to take a look at the code today.

Okay, fair enough. I'm going to test it again to make sure I didn't break something.

Sounds good @brndnmtthws I'll report what I find as well (hoping before noon).
Thanks for the work also!!

Okay I found the issue, if the app doesn't have health checks then it gets skipped for whatever reason.
Here is requestbin1:
{
  "container": {
    "type": "DOCKER",
    "docker": {
      "network": "BRIDGE",
      "image": "picoded/httpbin",
      "portMappings": [{"containerPort": 80, "hostPort": 0, "protocol": "tcp"}]
    }
  },
  "id": "requestbin1",
  "instances": 1,
  "cpus": 0.1,
  "mem": 32,
  "labels": {
    "HAPROXY_0_MODE": "http",
    "HAPROXY_0_VHOST": "requestbin1"
  },
  "healthChecks": [
    {
        "protocol": "HTTP",
        "path": "/delay/2",
        "gracePeriodSeconds": 3,
        "intervalSeconds": 10,
        "portIndex": 0,
        "timeoutSeconds": 5,
        "maxConsecutiveFailures": 3
    }
  ]
}

Here is requestbin2:
{
  "container": {
    "type": "DOCKER",
    "docker": {
      "network": "BRIDGE",
      "image": "picoded/httpbin",
      "portMappings": [{"containerPort": 80, "hostPort": 0, "protocol": "tcp"}]
    }
  },
  "id": "requestbin2",
  "instances": 1,
  "cpus": 0.1,
  "mem": 32,
  "labels": {
    "HAPROXY_0_MODE": "http",
    "HAPROXY_0_VHOST": "requestbin2"
  }
}

I modified the part here https://github.com/brndnmtthws/marathon-lb/blob/brndnmtthws/lb-improvements/marathon-update-haproxy.py#L1005-L1010 with the following code:
    # Convert into a list for easier consumption
    apps_list = list()
    for marathon_app in marathon_apps:
        logger.info("MARATHON_APP=%s" % marathon_app.services)
        for service in marathon_app.services.values():
            logger.info("SERVICE=%s" % service)
            if service.backends:
                logger.info("SERVICE_BACKENDS=%s" % service.backends)
                apps_list.append(service)
    logger.info("APPS_LIST=%s" % apps_list)
    return apps_list

When running with your branch and the modified code with extra logging:
servicerouter: MARATHON_APP={10000: MarathonService(u'/requestbin1', 10000)}
servicerouter: SERVICE=MarathonService(u'/requestbin1', 10000)
servicerouter: SERVICE_BACKENDS=set([MarathonBackend(u'<IP_ADDRESS>', 31393)])
servicerouter: MARATHON_APP={10001: MarathonService(u'/requestbin2', 10001)}
servicerouter: SERVICE=MarathonService(u'/requestbin2', 10001)
servicerouter: APPS_LIST=[MarathonService(u'/requestbin1', 10000)]

You can see that requestbin2 does not get appended to the apps_list list as the if service.backends skips it.
You can see the output from Marathon apps endpoint for both requestbin1 and requestbin2 apps: https://gist.github.com/scalp42/5163046000e9ed1bc055

Ah yes. I know why this is. I'll update the PR in a sec.

Haha @brndnmtthws amazing, I updated my comment with more details in case.

Also if I could make a suggestion please add the following to .gitignore:
templates/HA*

But still create the empty dir templates with .gitkeep inside it if it makes sense.
This is for the folks not using the Docker container (and makes it configuration management friendly).

My bad, templates/HA* is less preferred to templates but it's nitpicking at this point.
Thanks a lot for your work! Going to test it now.

Working perfectly :100:

I see what's happening. I was trying to call a method without the parens (()). I guess python isn't ruby.
Fixed now.

@brndnmtthws still happening :/
Let me try to figure it out as well.

Hmm, that might not be the intended behaviour. The idea behind --health-check is to use Marathon's health checks. The code you pointed to above is if you want to enable HAProxy's health checks independent of Marathon's.
I'm not suggesting that's intuitive, but that's currently how it's implemented.

I'm confused about this. The way I understand it is:
I have 2 apps:

APP A with health check HTTP /ping
APP B with no health checks

If I don't use the --health-check flag, I would assume the two following blocks in Haproxy:
backend APP1_10000
  balance roundrobin
  mode http
  server 192_168_99_112_31393 <IP_ADDRESS>:31393

backend APP2_10001
  balance roundrobin
  mode http
  server 192_168_99_112_31681 <IP_ADDRESS>:31681

If I use the --health-check, I would assume the two following blocks in Haproxy:
backend APP1_10000
  balance roundrobin
  mode http
  option  httpchk GET /ping
  timeout check 5s
  server 192_168_99_112_31393 <IP_ADDRESS>:31393

backend APP2_10001
  balance roundrobin
  mode http
  server 192_168_99_112_31681 <IP_ADDRESS>:31681

Could you please precise the difference between my understanding and your implementation?
If I don't use the and args.healthcheck fix, then wether I use the --health-check or not, the end result is the same for APP1 and APP2.

--health-check only applies to Marathon's health checks. In other words, if it's enabled, the app endpoints (backends) will only be exposed when Marathon reports the app instances as being healthy. If there are no health checks set in Marathon, the app's backends will be added to the LB pool regardless of their health state.
You can still use HAProxy's health checks regardless of Marathon's status, which is where the templates come into play.
Is that more clear?
I'll also update the --help info to clarify.

Oh that makes way more sense now gotcha:

the health checks specified through Marathon are always translated to Haproxy health checks unless you override HAPROXY_BACKEND_HTTP_HEALTHCHECK_OPTIONS to blank for example

when --health-check is specified, if Marathon fails a health check to one of the app, then it gets removed from the configuration file itself in Haproxy.

:+1:

That's right!
:bomb:'s away!

Huge :+1: again, much much appreciated.


RANGER-4954: Fix multiple vulnerabilities

What changes were proposed in this pull request?
RANGER-4954 Fix multiple vulnerabilities
CVE-2024-47561
CVE-2023-39410
CVE-2022-26612
CVE-2024-23454
CVE-2023-25613
CVE-2024-7254
CVE-2022-3510
CVE-2022-3509
CVE-2021-22570
CVE-2021-22569
CVE-2022-3171
CVE-2022-25857
CVE-2022-1471
CVE-2022-41854
CVE-2022-38752
CVE-2022-38751
CVE-2022-38750
CVE-2022-38749
CVE-2023-31418
CVE-2024-23450
CVE-2024-23444
CVE-2023-49921
CVE-2023-46673
CVE-2023-31419
CVE-2023-31417
CVE-2021-22144
CVE-2021-22135
CVE-2021-22134
CVE-2023-46589
CVE-2024-24549
CVE-2024-23672
CVE-2024-25710
CVE-2024-26308
CVE-2023-43642
CVE-2024-22201
CVE-2024-36114
CVE-2024-25638
CVE-2024-47554
CVE-2024-7254
How was this patch tested?
By running ./ranger_in_docker up and testing all the functionalities of Apache ranger

Thanks @FerArribas14 for reporting the CVEs, from the description this seems to be too many issues to be tracked in a single PR. Could you please split this into multiple PRs, maybe based on severity - like one for Critical CVEs and one for High CVEs. thanks
CC: @mneethiraj

@FerArribas14  As @kumaab  mentioned could you please list down the CVE's with severity and create small PR's if possible .Adding all the CVE's in one PR will be difficult to see the impact analysis while reviewing

@ramackri, @kumaab Perfect, I think it's a good idea.  All the vulnerabilities I have resolved are Critical and High. Therefore, I separate the PR into these two severities. Do you agree?


Address HIGH RISK data register security vulnerabilities

Description:
As a team, we should address security vulnerabilities before they become an issue. There are currently 14 high risk vulnerabilities identified.  https://github.com/bcgov/parks-data-register/security/dependabot
Acceptance Criteria:

[ ] each of the high priority vulnerabilities have been addressed/fixed

Development Checklist:

[x] Determine which vulnerabilities we want to tackled (note: thinking high)
[x] Come up with approach (i.e., does each vulnerability need a ticket in the backlog?)
[ ] ...

Dependencies

Blocked by
Blocking

Relevant documentation as reference
Definition of Ready

[ ] Acceptance criteria are included
[ ] Wireframes are included (if applicable)
[ ] Design / Solution is accepted by Product Owner (if applicable)
[ ] Dependencies are identified (technical, business, regulatory/policy)
[ ] Story has been estimated (under 13 pts)

Definition of Done

In progress:

[ ] Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
[ ] UI meets accessibility requirements
[ ] Unit tests are written
[ ] Work is traceable in GitHub
[ ] PR linked to ticket number
[ ] If needed/required - Dev adds flag/label to highlight any migration steps necessary prior to PROD deployment

Code review:

[ ] Code is peer reviewed and has passed CI/CD tests

QA:

[ ] Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
[ ] Code is potentially shippable to the production environment
[ ] Functional features have been tested and passed by QA
[ ] UI components tested by designer
[ ] Code is deployed to PROD when moved to 'done' column (unless requested otherwise by PO)

PO Review:

[ ] Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
[ ] Reviewed and approved by Product Owner

Notes:

@LindsayMacfarlane a good one to bring up at next refinement.

All of the security vulnerabilities deal with dependencies of dependencies being out of date. This means we cannot directly address the security vulnerabilities, only update our immediate dependencies to their latest versions and hope that they in turn are addressing the security vulnerabilities. Packages have been updated to the latest version as of https://github.com/bcgov/parks-data-register/pull/550 and https://github.com/bcgov/parks-data-register/pull/551.

@cameronpettit could you please let me know what sort of testing is best for this ticket? Does the changes impact the project code or is it just github settings only?

Thank you @cameronpettit

Tested on TEST: Passed

Smoke tested the test Data registry and everything looks good.
Moving to PO review.

Thank you @cameronpettit for addressing the immediate dependencies. And thank you @manuji for conducting the smoke test.
Approved. Moving to done.


Comparison between projects

Dear Sirs,
I'm looking around for a way to transmit both video and RC Mavlink data over 2.4 Ghz, found many projects about this but cannot get a starting point:
From this document https://dronebridge.gitbook.io/docs/comparison they put comparison on:
svpcoms WifiBroadcast (basic TX-RX)
DroneBridge (system design with library)
OpenHD (follow-up on EZ-WifiBroadcast)
EZ-Wifibroadcast (not under development)
But looking at code the most updated seems that svpcoms WifiBroadcast and is able to manage both video and Mavlink.
Dronebridge is still developed?

All of them can do MAVLink telemetry. MAVLink RC is currently limited to 8 channels as far as I kno. With DroneBridge you could also easily write your own python script to read & transmit the data using the library. But nothing wrong with using svpcoms implementation.
DroneBridge is under active development, but not as intense at the moment since I got other things on my plate as well.


lnd 0.8.1 -> lnd 0.9.0

Motivation for this change
Changelog at https://github.com/lightningnetwork/lnd/releases
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions (ArchLinux)

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Thanks!


add function verifyJwtWithoutTimeChecking

See https://github.com/envoyproxy/envoy/pull/10753/files/4b973ea71e104cb42488341982ea58545d91d4f4..6d27fff1721c92f6938fbe88796e698dc7d04bff

Could you please sign CLA?

signed

@googlebot I signed it!

You were using different email in your git commit.  It doens't match your email in github account.


document actInsideInputWithModifier  useKeyPress option

Please describe the feature that you want to propose
Hi.
In my project, I use a UseKeyPress to trigger when CTRL + A is pressed to select all nodes & edges.
Also, my nodes contains inputs to enter some text.
The problem is, when I use CTRL + A on my input to select all the text inside, the UseKeyPress is triggered so it selects all my nodes.
Can we get a way to avoid useKeyPress to be triggered if a certain element is focused ? I tried the "nokey" className but it doesn't work in this case.
Thank you !

Added this to the documentation: https://reactflow.dev/api-reference/hooks/use-key-press#options


Update node modules, fix CVE-2018-3728

We are in the process of switching to this library for the hubot-home-assistant module, and noticed that is has a vulnerable verison of hoek in it, via the request module. This PR resolves it by catching it up.
Related: https://nvd.nist.gov/vuln/detail/CVE-2018-3728

/cc @goyney ; Let me know if there's anything else you'd like to see with this. 😄

Thanks, I'll publish an update to NPM later this evening.

@goyney Thank you!

0.1.5 was just published. Thanks!


Fix CONTRIBUTING documentation

This PR updates the CONTRIBUTING.md file so given commands are actually valid and tests can be run.
The bundler commands given contained deprecated arguments that no longer worked.

@stevenpost thanks for the PR! The file is actually rendered from a template: https://github.com/voxpupuli/modulesync_config/blob/master/moduleroot/.github/CONTRIBUTING.md.erb
could you please send the PR to the template?

@stevenpost thanks for the PR! The file is actually rendered from a template: https://github.com/voxpupuli/modulesync_config/blob/master/moduleroot/.github/CONTRIBUTING.md.erb
could you please send the PR to the template?

I'll have a look, will reference it here and then close this PR.

Superseeded by https://github.com/voxpupuli/modulesync_config/pull/881
Transposing the changes was easy enough:
git format-patch master..
sed -i 's;\.github/CONTRIBUTING.md;moduleroot/.github/CONTRIBUTING.md.erb;g' 000*
cd ../modulesync_config
git am ../puppet-mongodb/000*


updated src

Added a new algorithm that counts the number of distinct elements in an array

I'm extremely sorry.  I wanted to add a few algorithms. But, I cant figure
out how to resolve build and test with maven issue.
Im facing difficulty in that.
On Wed, 6 Oct, 2021, 12:08 pm Pradyuman Dixit, @.***>
wrote:

Closing this for now, please feel free to reopen later if you want to work
on this
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Pradyuman7/Algorithmics/pull/27#issuecomment-935580010,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ATL4OQ43JDCTGGF5ZNIZ3HDUFPVGJANCNFSM5FHSHHJA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@Divi2701 no worries. From what I can see the issue is the typo in the file name CountDistict instead of CountDistinct which is the name of the public class. If you fix that typo, that should fix the build.
Also, please write some tests for your code.


Nuget Installation Failing

In my project I have the latest Newtonsoft.Json (>= 10.0.3)
The nuget failed to install because of version conflict detected for Newtonsoft.Json.
Version conflict detected for Newtonsoft.Json.
// -> Microsoft.Azure.Management.Cdn (>= 2.1.0) -> Microsoft.Rest.ClientRuntime (>= 2.3.1) -> Newtonsoft.Json (>= 9.0.1).
Installing NuGet package Microsoft.Azure.Management.Cdn 2.1.0.
Committing restore...
With Best Regards,

Not sure if this is still an issue for you but its been a long time. We're removing our dependency on JSON.NET in future releases so we expect this problem to go away over time. Let us know if there still issues you are hitting by opening up a new issue. Thanks!


Curl: Check all responses for protected cookies

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

The response from a URL protected by Cloudflare may only provide a relevant cookie on the first response but #curl_http_content_headers_and_checksum only returns the headers of the final response. In this scenario, #curl_check_http_content isn't able to properly detect the protected URL and this is surfaced as an error instead of skipping the URL (see https://github.com/Homebrew/homebrew-cask/pull/124277).
This PR resolves the issue by including the array of response hashes in the return value from #curl_http_content_headers_and_checksum, so we can check all the responses in #curl_check_http_content.
Besides that, this also renames the :status key in the return hash from #curl_http_content_headers_and_checksum to :status_code. This aligns the key name with the return hash from #parse_curl_response and technically allows us to pass either hash to the #url_protected_by_* methods (as they both contain :status_code and :headers in the expected format. This felt like a nicer way of approaching the switch from the details hash to a response hash.

Review period skipped due to critical label.


Review dotnet pack documentation

https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-pack

Looks good overall- @KathleenDollard seems to have a change that will fix #9354. @KathleenDollard is that change ready to merge?

#13281 on this article is awaiting an answer.

Reading through the article everything looks good to me. There are 2 open issues but #13281 belongs on another area path (see my comment on that PR) and #17704 looks taken care of.


rename recordID to id

also we should consider how id handling is different in android vs iOS (in ios record id's are somewhat transient)

could you expand on this? you want to rename the recordID field to id on both platforms?

Ya. Originally I named it recordID to parallel the terminology of iOS AddressBook. But this feels arbitrary, and its weird in camel case should it actually be recordId? But then in Contacts framework they use "identifier" and in android it is "id" I believe - so recordID feels antiquated.
This would be a major breaking change but migration would be trivial. Thoughts?

trivial agreed. I would just change this line https://github.com/rt2zz/react-native-contacts/blob/master/android/src/main/java/com/rt2zz/reactnativecontacts/ContactsManager.java#L77

just release a new major release with this


⚠ OhDee has degraded performance

In 1b8d8e0, OhDee (https://ohdee.com.au) experienced degraded performance:

HTTP code: 200
Response time: 8057 ms

Resolved: OhDee performance has improved in 8a21a29.


Force flush in BIO_free.

CLA: Trivial
Fixes: #14517
Signed-off-by: Randall S. Becker<EMAIL_ADDRESS>
Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

Uhmmmm, CIs are going belly up

Uhmmmm, CIs are going belly up

On a one-liner?

The CI crashes are weird, and indicate that flushing has bugs in BIO

The CI crashes are weird, and indicate that flushing has bugs in BIO

I just ran tests on both NonStop and Windows x86 under gcc/cygwin and am not seeing any issues. BIO_ctrl has all the guards in place to prevent null problems.

The CI crashes are weird, and indicate that flushing has bugs in BIO

I just ran tests on both NonStop and Windows x86 under gcc/cygwin and am not seeing any issues. BIO_ctrl has all the guards in place to prevent null problems.

There may be a possible problem if parg is null in BIO_ctrl, but that seems like a legitimate use of the argument. BIO_flush is a simple invocation and has a guard test on the file not being null evaluated by BIO_ctrl, unless there is a timing thing that the file is not null but is in a closed state by the time BIO_flush is invoked. Can this condition be tested for in code inside BIO_ctrl?

The multiple CI tests are failing for an unrelated reason.
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.

The multiple CI tests are failing for an unrelated reason.
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.

That's hardly the actual reason

The multiple CI tests are failing for an unrelated reason.
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.

That's hardly the actual reason

Just trying to help ;)

Found the issue, this is a quick fix that should be good enough:
diff --git a/crypto/bio/bio_lib.c b/crypto/bio/bio_lib.c
index da61544937..1d317248cc 100644
--- a/crypto/bio/bio_lib.c
+++ b/crypto/bio/bio_lib.c
@@ -114,7 +114,9 @@ int BIO_free(BIO *a)
     if (a == NULL)
         return 0;

-    (void) BIO_flush(a);
+    /* It's only safe to do this on source BIOs */
+    if (a->next_bio == NULL)
+        (void) BIO_flush(a);

     if (CRYPTO_DOWN_REF(&a->references, &ret, a->lock) <= 0)
         return 0;

What happens is that if this is a filter BIO, whose underlying source BIO has been freed first, running anything that goes down the BIO chain will crash.  BIO_flush() is one of those.

Found the issue, this is a quick fix that should be good enough:
diff --git a/crypto/bio/bio_lib.c b/crypto/bio/bio_lib.c
index da61544937..1d317248cc 100644
--- a/crypto/bio/bio_lib.c
+++ b/crypto/bio/bio_lib.c
@@ -114,7 +114,9 @@ int BIO_free(BIO *a)
     if (a == NULL)
         return 0;

-    (void) BIO_flush(a);
+    /* It's only safe to do this on source BIOs */
+    if (a->next_bio == NULL)
+        (void) BIO_flush(a);

     if (CRYPTO_DOWN_REF(&a->references, &ret, a->lock) <= 0)
         return 0;

What happens is that if this is a filter BIO, whose underlying source BIO has been freed first, running anything that goes down the BIO chain will crash. BIO_flush() is one of those.

I feel bad about this. I had that code there in the first place and thought it was unnecessary because of the checks in BIO_ctrl.

Found the issue, this is a quick fix that should be good enough:
diff --git a/crypto/bio/bio_lib.c b/crypto/bio/bio_lib.c
index da61544937..1d317248cc 100644
--- a/crypto/bio/bio_lib.c
+++ b/crypto/bio/bio_lib.c
@@ -114,7 +114,9 @@ int BIO_free(BIO *a)
     if (a == NULL)
         return 0;

-    (void) BIO_flush(a);
+    /* It's only safe to do this on source BIOs */
+    if (a->next_bio == NULL)
+        (void) BIO_flush(a);

     if (CRYPTO_DOWN_REF(&a->references, &ret, a->lock) <= 0)
         return 0;

What happens is that if this is a filter BIO, whose underlying source BIO has been freed first, running anything that goes down the BIO chain will crash. BIO_flush() is one of those.

I feel bad about this. I had that code there in the first place and thought it was unnecessary because of the checks in BIO_ctrl.

@levitte  Shouldn't it be != NULL?

I feel bad about this. I had that code there in the first place and thought it was unnecessary because of the checks in BIO_ctrl.

BIO_ctrl has the potential of traveling down the BIO chain, by design.  Unfortunately, we have the situation in apps/dgst.c with this chain: bmd -> in.  At the end of the main function, in gets freed before bmd, leaving a dangling next_bio in the latter.  KABOOM
(resolving this properly has implications that I think goes beyond this PR)

I am worried about the effect this might have on input BIOs. From the manpage: For input streams associated with seekable files (e.g., disk files, but not pipes or terminals), fflush() discards any buffered data  that  has been fetched from the underlying file, but has not been consumed by the application. Not sure if that means the file pointer will be moved and the file will be just re-read on subsequent reads. In that case it is just a minor inefficiency. Otherwise it would be a problem.

@t8m, good point.  Perhaps that should only be done when the flag BIO_CLOSE is set.

@t8m, good point. Perhaps that should only be done when the flag BIO_CLOSE is set.

I am fine with dropping this PR if it's going to present problems. PR #14623 should fix the root problem.

I am fine with dropping this PR if it's going to present problems. PR #14623 should fix the root problem.

It might be wise at this point, as we seem to be heading into a rabbit hole.  I merged #14623 a minute ago

Fixed by #14623


not getting output and not printing

i am working in windows with pycharm. its not giving output. showing errors

this is what iam getting. lot of times i am getting exception only

these are the final module errors. what to do now


Can't import models module

Hi there,
first of all thank you so much for taking the time and publishing this library, appreciate it a lot since peewee doesn't come with automatic migrations included.
I'm trying to use the cli to migrate my model, but load_models raises an Exception, saying that it can't import the models module (the one at the --auto path).
This is the failing command:
$ pw_migrate create --auto ~/myproject/model/ --database "postgres://zudibytvipthsa:c8c2d7f0blabla" --directory ~/myproject/model/ mymigration
Could it be that peewee_migrate doesn't resolve the database connection URL with playhouse.db_url.connect but expects something else instead? This isn't really clear from the help text.
My model module is structured as follows:
model/
    basemodel.py  <-- containing:
    class BaseModel(Model):
        class Meta:
             database = db

    .. more modules that inherit from BaseModel

I would appreciate getting some hints as to what might be causing this :)

first you could take a look at #16 because of your BaseModel.
i don't really know postgres, but the url should work like explained in the playhouse docs.
i think the command fails because it can't import your module:
 pw_migrate create --auto=model.basemodel --database="postgres://zudibytvipthsa:c8c2d7f0blabla" --directory="~/myproject/model/" mymigration
this could work. i think you have to make sure that you can import the models with python like a normal module from your working directory.

You can try adding the following lines to the beginning of the migration file:
import os
import sys
sys.path.append(os.path.dirname("__file__"))

You can try adding the following lines to the beginning of the migration file:
import os
import sys
sys.path.append(os.path.dirname("__file__"))

Whew, answers coming in 4 years later 😁 Appreciate it, but I've moved on

Whew, answers coming in 4 years later 😁 Appreciate it, but I've moved on

Whew, answers coming in 4 years later 😁 Appreciate it, but I've moved on

They're not just for you :slightly_smiling_face:


Fix cleanup script missing scenario

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

There were dataproc instances leakiing named like beam-loadtests-py-cmb-flink-batch-6642937942, which is not included in the current list
Also make bigquery dataset cleaner work for macOS. There are lots of temp dataset permission denied, possibly created by other service account. Will need a manual run of this script from admin of the project

R: @damccorm @volatilemolotov


Static library not found

Bintray has been deprecated, therefore i am unable to download the static libraries for ios and android. Is there an alternate source for this?

https://github.com/hyperledger/aries-mobile-agent-xamarin/issues/44#issuecomment-920930878


Im 108 change button when user logged in

I don't know if it's the frontend or backend, but this ticket 108 is acting really bugging for me. I think it worked once, and then I deleted my cookies and restarted the front and back ends and it's giving me undefined for a name and no login button.


Command line interface

Hi ! First of all thanks a lot for this awesome tool.
I'm using it to generate waveforms for some documentation. I currently have some json wave files in a directory. In order to process them with latex, I need pdf. I can easily go from svg to pdf (which is great because everything is vector) with inkscape. The only thing I can't automate is the export of json to svg. I currently open the editor and click "export svg".
Hence my question : would it be possible to imagine a command line interface doing something like this ?
./wavedrom --in wave.json --export-svg --out wave.svg
Thanks for your time !

Hi @Bamban . Thank you for the kind words. Here is CLI version of wavedrom https://github.com/wavedrom/cli Please try.

CLI is really a great idea!
Any chance to get it implemented directly into the standalone waveformeditor, since the editor has all needed options already implemented and especially since PhantomJS project has been suspended?
Thanks a lot spending your time!

Yes, It is possible since chromium has headless mode now: https://developers.google.com/web/updates/2017/04/headless-chrome
And if NWJS supports it I will implement CLI for it.
https://github.com/nwjs/nw.js/issues/769


Using binding inside an async/go block is not generally safe

Consider the following example1:
(def ^:dynamic *foo* :original)

(defn -main [& _]
  (let [c (clojure.core.async/to-chan [42])]
    (clojure.core.async/go
      (binding [*foo* :rebound]
        (clojure.core.async/<! c)
        (println "done."))))

  (Thread/sleep 5000))

The go macro transforms its body to create a state machine that can be paused/resumed on parking operations. That state machine is then executed on a worker thread pool. Since the <! operation in the above example causes the state machine to park (i.e., yield), the worker thread that entered the binding block and pushed the new *foo* value onto its dynamic-bindings stack might not be the same thread that resumes the state machine (after the <! operation completes), which means that the dynamic-bindings pop operation at the end of the binding block can fail loudly:
Exception in thread "async-dispatch-3" java.lang.IllegalStateException: Pop without matching push

This is a known issue. The issue description says that it only applies to ahead-of-time-compiled code (e.g., apps run from an UberJAR)—but it's not clear to me why it wouldn't also apply to non-AOT-compiled code.
We've seen these errors cause Waiter to crash in our production environment. We should try our best to avoid using binding blocks within async/go blocks. If we can't avoid it, then we must ensure that there are no parking actions within the binding block.

[1]: Example adapted from the proof-of-concept in https://dev.clojure.org/jira/browse/ASYNC-170.


debug pvrcnn_ssl.py

how can i debug it?when i debug, it always wait

Hi @zscya
You can save the intermediate variable by the first debug(e.g., det_annos, aug_det_annos, and dual_threshold in here), and you can directly load these files in the subsequent debug for fast running.

thank you ,i try it


fix: fix typo in image inspect cli

Signed-off-by: Wei Fu<EMAIL_ADDRESS>
1.Describe what this PR did
Fix typo in image inspect command
2.Does this pull request fix one issue?
NONE
3.Describe how you did it
NONE
4.Describe how to verify it
NONE
5.Special notes for reviews
NONE

LGTM


Integration with tools

Could the agent be integrated with custom tools (i.e. search)? It would be nice to see such example...

Could the agent be integrated with custom tools (i.e. search)? It would be nice to see such example...

Thanks for your advice, we wil consider to try on that!


⚠ Jenkins Plugins Artifactory has degraded performance

In 7e186b0, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1273 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 2933154 after 12 minutes.


RHEL 6.7 baseurl for yum repo expands incorrectly

On at least Red Hat Enterprise Linux Server release 6.7 (Santiago): The baseurl in /etc/yum.repos.d/gitlab_official.repo exands to:
https://packages.gitlab.com/gitlab/gitlab-ce/el/6Server/x86_64/repodata/repomd.xml - which is incorrect.
It should expand to:
https://packages.gitlab.com/gitlab/gitlab-ce/el/6/x86_64/repodata/repomd.xml
Tested on v1.7.0

Probably related to #41

Could you please verify if this issue still exists with the latest release 1.7.1?

As I didn't hear back any news on this issue, I'll close it right now. Feel free to open a new one or reopen this.


[SPARK-50199][PYTHON][TESTS] Use Spark 3.4.4 instead of 3.0.1 in test_install_spark

What changes were proposed in this pull request?
This PR aims to use Spark 3.4.4 instead of 3.0.1 in test_install_spark.
Since Spark 3.4.4 is the End-Of-Life release, it will be in dlcdc, archive, all dist channel until Apache Spark 4.0 release. Previously, 3.0.1 exists only in archive.
Why are the changes needed?
To reduce the flakiness.

https://github.com/apache/spark/actions/runs/11623974780/job/32371883850

urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>
ERROR
test_package_name (pyspark.tests.test_install_spark.SparkInstallationTestCase) ... Trying to download Spark spark-3.0.1 from [https://dlcdn.apache.org/, https://archive.apache.org/dist, https://dist.apache.org/repos/dist/release]
Downloading spark-3.0.1 for Hadoop hadoop3.2 from:
- https://dlcdn.apache.org//spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz
Failed to download spark-3.0.1 for Hadoop hadoop3.2 from https://dlcdn.apache.org//spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz:
Downloading spark-3.0.1 for Hadoop hadoop3.2 from:
- https://archive.apache.org/dist/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz
Failed to download spark-3.0.1 for Hadoop hadoop3.2 from https://archive.apache.org/dist/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz:
Downloading spark-3.0.1 for Hadoop hadoop3.2 from:
- https://dist.apache.org/repos/dist/release/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz
Failed to download spark-3.0.1 for Hadoop hadoop3.2 from https://dist.apache.org/repos/dist/release/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz:
ok

AFTER
test_install_spark (pyspark.tests.test_install_spark.SparkInstallationTestCase) ... Trying to download Spark spark-3.4.4 from [https://dlcdn.apache.org/, https://archive.apache.org/dist, https://dist.apache.org/repos/dist/release]
Downloading spark-3.4.4 for Hadoop hadoop3 from:
- https://dlcdn.apache.org//spark/spark-3.4.4/spark-3.4.4-bin-hadoop3.tgz
Downloaded 1048576 of 388988563 bytes (0.27%)
...

Since Spark 3.4.4 is the EOL version, it will be in download.apache.org until Apache Spark 4.0.0 release.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass the CIs.
Was this patch authored or co-authored using generative AI tooling?
No.

cc @HyukjinKwon

Could you review this, @xinrong-meng ?

Yeah, we have to change this.

Thank you, @HyukjinKwon !

Merged to master/3.5.


Makefile: Incomplete 'make clean' command and 'make pdf' tweak

In Makefile:

Calling make clean doesn't have a cd challenges_assignment; make clean call to remove assignments created from the challenge master
make clean in challenges_assignment, exercises_assignment, and exercises_solution does not remove the generated "images" folder
make pdf does not have a 'main' call before calling jb build which would enable changes between Makefile commands to not be presented in the pdf

1 and 3 resolved by d9ee477; clearing the images folder doesn't seem as important to me. The assignment files should update, but image files will remain relatively static.


Deprecate Track1 Azure Query Monitor SDKs

Tracking issue to deprecate the following packages in favor of https://www.nuget.org/packages/Azure.Monitor.Query/:

[x] https://www.nuget.org/packages/Microsoft.Azure.OperationalInsights/
[ ] https://www.nuget.org/packages/Microsoft.Azure.ApplicationInsights.Query/

Microsoft.Azure.OperationalInsights was deprecated on Jan. 6th.

Microsoft.Azure.ApplicationInsights.Query was deprecated on March 16th.


Azure Firewall Workbook - garbage data

Describe the bug
The Network and Rule Hitcount (SUM) and Application Rule Hitcount (SUM) data is pretty useless because the values on the Y axis is changing when time range is changed. Consider the comparison of charts from the last 24 hours vs the last 48 hours.
The last 24 hours:

The last 48 hours:

Now compare the last 24 hours on both chart, they are inconsistent. One can see that the values on Y axis are increasing when time range is selected.
The chart seems to be aggregating values in some unknown time buckets making the values on the Y axis not understandable. The readings on the Y axis should be consistent for the given time period regardless of the time range selected.

The issue is also being triggered by simple browser resize, please see this GIF:

@gumoden Hey Gustavo, could you please help take a look at this.

@nosalan This is a built in Metric that is being used in the Firewall workbook, however if you want to define the time buckets, you can choose the Time Granularity parameter as shown in the below image to solve this.


feat(lang): support include other git repo for envd functions/variables

Signed-off-by: Keming<EMAIL_ADDRESS>
close #559

FAQ

Why not load or import?

load is starlark keyword, import has been reserved by starlark.

/lgtm
testing muniu-bot

/retest

/retest

/test all

/lgtm

Please test the bot in #812


Excessive logging for publisher

This seems to be slowing down Znode sync in particular.

Fixed in https://github.com/zcoinofficial/zcoin/commit/bca2279fdaabf6b333e44ed20f97a2d1e5ecea9d


No list functionality for single tags

Nearly all BBCode engines I've used in the past use the following format:
[list]
[*]List item 1
[*]List item 2
[*]List item 3
[/list]

Same as:

List item 1

List item 2

List item 3

But this is impossible with the parser in its current set up. I can have a look to alter it one day, but I don't know if I'll find the time. Can you help with this?

The easiest way is to implement this is when parsing and encountering a [] tag and the previous tag is a [] tag, we see this as the end tag for the previous tag and the start tag of the current. We also need to see [/list] tag as the end tag for [*] tag.
This is not such a nice solution, but probably the easiest one to implement.

@svenslaggare agreed, checking that anything following a [*] tag with an open close bracket regardless of content will probably do it.


Set retries default to 2

As defined in #403

The kafka-retries parameter in #403 also fixed my problems.
But ist was not the default option in 0.8.2 when running in standalone mode (default is still 0).
Never mind, I found it, but it would be very helpful if it was also listed in the documentation.
If a pull-request would be desired here, I could try my hand at it.


Stats about purchased weapons (on export maybe)

I don't know weather this is even possible but I would love to see stats about what weapon was bought each round. I know there's a twitter account who posts stats about weapon purchased at huge events (unfortunately his name slipped my mind but I will add it if I can find him again)

A new major version has been released, closing the issue 🧹


A few things I found in the Android Engine code

What's up doc?
I was able to take a look at the android shell using Android Studio and it pointed some warnings to me that I thought that made sense.

In FlutterActivity.java and FlutterActivityDelegate.java, GET_ACTIVITIES is used as part of flags for reading METADA, but AS says it is being miss used. (screenshot attached).
As far as I can tel, the GET_ACTIVITIES is not needed.

At PlatformPlugin.java there is a switch statement with this code..

            case HEAVY_IMPACT:
                // HapticFeedbackConstants.CONTEXT_CLICK from API level 23.
                view.performHapticFeedback(6);
                break;
            case SELECTION_CLICK:
                view.performHapticFeedback(HapticFeedbackConstants.CLOCK_TICK);
                break;

I understood the magic number 6 being used to replace the constant HapticFeedbackConstants.CONTEXT_CLICK that only exist in API +23, but
HapticFeedbackConstants.CLOCK_TICK is API +21, shouldn't this have the same handling? I.e., using the literal instead of the constant.

also in PlatformPlugin.java there is this block

        if (overlaysToShow.size() == 0) {
            enabledOverlays |= View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY;
        }

that is not SDK_INT checked, with SYSTEM_UI_FLAG_IMMERSIVE_STICKY being is API +19, this would result a crash for older phones, right?

In FlutterActivityAndFragmentDelegate .java there is this code

 if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
     flutterSplashView.setId(View.generateViewId());
  } else {
    // TODO(mattcarroll): Find a better solution to this ID. This is a random, static ID.
    // It might conflict with other Views, and it means that only a single FlutterSplashView
    // can exist in a View hierarchy at one time.
    flutterSplashView.setId(486947586);
  }

The TODO can be removed be doing
flutterSplashView.setId(ViewCompat.generateViewId());
cc: @matthew-carroll

@BugsBunnyBR thanks for looking through the code. I'll add this ticket to the embedding refactor project and take a look at each issue you mentioned as soon as I have time.

Hi @BugsBunnyBR I'm finally getting around to reviewing this. Here is what I'm seeing.

GET_ACTIVITIES issue

In my Android Studio I'm not seeing any warning about GET_ACTIVITIES. Also, the docs say:
    /**
     * {@link PackageInfo} flag: return information about
     * activities in the package in {@link PackageInfo#activities}.
     */
    public static final int GET_ACTIVITIES              = 0x00000001;

So I do think we want GET_ACTIVITIES so that we limit the meta-data search to Activitys. Does that sound right?

Haptic values and API versions

I'm going to make 2 changes to this. First, I'm going to add if-statements around the haptic invocations so that we don't pass values to older Android versions that won't understand them. Second, I'm going to change the magic number of the real declaration - the reason we used a magic number wasn't actually about backwards compatibility, I think it was for forwards compatibility back when Flutter was building against old Android versions.

SYSTEM_UI_FLAG_IMMERSIVE_STICKY API version

I'm not sure if passing a newer value would crash or just be ignored, but I'll add an API guard in the if-statement.

Use ViewCompat for generating an ID

Unfortunately, I don't think we can use anything in ViewCompat because we need to minimize all possible library dependencies for binary size and future compatibility purposes.

About GET_ACTIVITIES
When I create a project MyApplication inside /git/engine/shell/platform/
and a sync link to android/io
cd ~/git/engine/shell/platform/MyApplication/app/src/main/java
ln -s ../../../../../android/io

and open MyApplication in Android Studio it does show the warning to my, the engine code does not compile due the missing dependencies, but it still shows the warning. (I am using the newest version of Android Studio 3.5.2)
When I look at other libraries for android, like facebook sdk,
and MixPanel there is no GET_ACTIVITIES flag there, so I guess it could be removed as suggested by the IDE if there is not side effect to what it is supposed to be doing.
About ViewCompat.
Even though you may wish to minimize the number of dependencies, there is something to take in consideration, that is, the value you get when using that given library. The engine code can become easier to read and keep by avoiding a number of version checks (if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT)) and the lib offers options some things that are not available in a given platform version (ViewCompat.generateViewId()). The engine depends of android_support_compat and other android support libraries and View compat is part of the set of libraries provided in the set of "support libraries"  and any "future compatibility issue" will be reduced because it is a COMPAT library as because wraps the diffs for you when possible.

The GET_ACTIVITIES flag has been removed: https://github.com/flutter/engine/pull/13932
WRT to ViewCompat, the Flutter team has had many discussions around dependencies. It would be great from a development standpoint to depend on various libraries. Unfortunately, managing the implications of those dependencies in terms of version resolution is intractable at this point in time. Perhaps at some point versioning will be at a place where Flutter can reasonably depend upon Android libraries, but for the foreseeable future the only such libraries are Fragment, for add-to-app, and Lifecycle, for plugins.

Closing this for now. We can't bring on additional transitive dependencies on the engine at the time.


Deploy the Website of this repo

Is your feature request related to a problem? Please describe.
I can see that the website related to this repo is not deployed anywhere.
Describe the solution you'd like
Deploy the website on cloud using Render since Heroku is now paid.

Assign this issue to me under JWOC

Hey @utkarsh006, we do not need this rn.


Feature Request: Reorderable Folders and watchers.

Is it possible to allow us to drag and drop the items in our watcher folders and the folders themselves?

Thanks for this suggestion @TRLOJason I'd love to implement drag and drop one day, but in the meantime you can change the folder a watcher belongs to using the dropdown under "Assigned Folder" in a watcher's settings.


Crash on load

Hello I am getting this crash Randomly.
crash-2023-07-23_09.56.16-fml.txt
When I restart the game, it loads normally. The mods that error also aren't always the same. I a not sure if this is your mod, but it is the only one that alters loading that I have insalled. Thanks in advance

Please remove Smooth Boot (Reloaded) and see if the issue still occur, if it still, open a new issue.


Add YAML metadata to front matter to identify preview modules

Summary of the new feature / enhancement
In reference to: https://dev.azure.com/ceapex/Engineering/_workitems/edit/404408
We need to define a meta data key that PlatyPS can render to yaml -- perhaps maml. Then OPS would use this information to render the appropriate preview disclaimers on the web pages.
Proposed technical implementation details (optional)
No response

How do you want PlatyPS to determine that the module is prerelease (-AsPreview switch, 0.x version, sematic version with prerelease)?
Which files should have the metadata (cmdlets, module, both)?
What should the metadata be? (eg ispreview: true|false)

@mikefrobbins We need to talk with Damien about how this will be used, where the metadata belongs (cmdlet, module, etc), and how it should be rendered on the Docs site.
@adityapatwardhan we need to settle on a schema for this feature.


Issue with box layout in Firefox.

Yes some of us still use :)  While it works in Chrome or Chrome based browsers I've tested.  It has issues in firefox as the box used to contain the sample and the scroll bar are misplaced on the page (see screenshot) This is using Firefox 108.1 (latest at the time of writing) On Windows 11

/cib


2.6-beta1 does not open browser tab on start on a Mac

This is on OS X 10.9.2.
Also, the application does not offer a file menu anymore that offers to open a new browser tab.
Workaround: Manually opening <IP_ADDRESS>:3333 works.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/2132645-2-6-beta1-does-not-open-browser-tab-on-start?utm_campaign=plugin&utm_content=tracker%2F32795&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F32795&utm_medium=issues&utm_source=github).

Fixed by c7aa25ccd3f2685bfe655ee3b689c3c47b5ae5cc from PR #1074


2.x compile fixes for asio standalone

Hi,
First off I want to say thanks so much for #176, before that I had my own fork that had removed boost as a dependency and I'm happy to ditch that.
These are the two minor things I had to do in order to get the 2.x branch compiling with VS 2019 / v142, I just wanted to let the maintainers know, and that otherwise very happy that this library exists in a form based on standalone asio.

Merged as https://github.com/socketio/socket.io-client-cpp/commit/c09221f357effe1a5a0fc0e7d7902eba1ab0484d and https://github.com/socketio/socket.io-client-cpp/commit/168ce9d10b4ac667c43fe16b4cf530f6a3749235. Thanks a lot :+1:


Cannot see config from localhost

Information

https://whatismybrowser.com/w/Q3N5RFA
Help request
Unable to see the config website.
Problem

When I run npm run dev I get a correct output. When I visit http://localhost:8080 I see only the text "2021"
What I have tried

I can briefly see the website being loaded but then it is covered up.
Screenshots

Actually noticed this myself yesterday -- this is due to a document.write being used in the Community website that we use as a template. I have a PR open upstream which hopefully should be merged/deployed today.

Thanks for the update. I pulled the latest version and did npm ci but am still having this issue.

Ah yeah, apologies, this fix hasn't gone out yet upstream -- we're planning to deploy today 🤞

Fix has been shipped upstream


Python syntax error (Python compilation error) does not show on the operator's result panel

When a Python UDF has some syntax error, it may pass the workflow compilation but it will fail during the executor initialization phase, as the Python will fail to compile the code. This error is then shown as the general workflow error and the user cannot see which Python UDF caused the issue. This is not very user-friendly.

Fixed by #2618


Create a landing page

Create a landing page that stays within the scope of gurunimbus

.take

.take


the parameter named "new" in NIFFS_rename is not compatible with C++

including niffs_api.h in a C++ project has a problem due to the parameter named "new" in NIFFS_rename(niffs *fs, char *old, char *new);
suggest renaming it

done


Support for fixed process on timer (Idle/Fixed mode) and a small change for more uniform ticks

Implemented support for fixed process on timer
This is useful for cooldowns without the influence from framerate.
The implementation is similar to animation player (idle/fixed mode), where idle mode = old timer, fixed mode = fixed process implementation:
Example of the new behaviour with a stream of bullets with timers for each shoot on different frame rates:
https://gfycat.com/HeartyImpressiveIndiancow

A change for more uniform ticks on fixed/idle
without this fix:

with this fix:

The change is:

  	time_left -= get_process_delta_time();

		if (time_left<0) {
			if (!one_shot)
				time_left=wait_time+time_left; //this line changed for: time_left = wait_time;
			else
				stop();
			emit_signal("timeout");
		}

The reason for the change is:
With time_left=wait_time+time_left; method: the algorithm takes on account the remaining ms from last tick for the next one, so not all ticks are the same
With time_left = wait_time; method: the algorithm discards the remaining ms, so all the ticks are the same
Example of the old behaviour with a timer of 0.04 ms on a fixed framerate of 60 (~0,016 ms):

0,0
0,016
0,032
0,048
----- TICK on 4 frames
0,008
0,024
0,040
----- TICK on 3 frames
0,0
0,016
0,032
0,048
----- TICK on 4 frames

So every second tick is more fastest, resulting on non uniform behaviour between all ticks, On the fix, all the ticks start with the same value, so is always the same result (4 frames)

nice contribution, thanks!


Point shapefiles and flatgeobuffs are treated as polygons.

When I upload files with points, they don't display because it appears that the data are treated as polygons, not points. If edit the style like this, it renders:

Would be good if SeaSketch recognized it as a point file.

Duplicate of #474


API key verification

GPT API key verification:

Verification done by making a GET request to the usage endpoint
Date param is required

The actual date doesn't seem to matter, tested using random dates from 2020 to 2023

Initial testing done by using postman

Returns error when using an invalid key

Returns a response when using valid key

Missing case:

Unsure what the response will be if the api key doesn't have any funds or has outstanding bills

LGTM


node-redis throw syntax error on FT.SEARCH query

Description
Expected result
https://redis.io/docs/interact/search-and-query/search/vectors/
FT.SEARCH idx "*=>[KNN 10 @vec $BLOB]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" DIALECT 2
Actual result
async function searchSimilar(vector: number[]) {
  const query = "*=>[KNN 10 @embedding $BLOB]";

  const options = {
    PARAMS: {
      BLOB: Buffer.from(new Float32Array(vector).buffer),
      DIALECT: 2,
    },
  };

  const result = await client.ft.search("idx:answers", query, options);

  return result;
}

Node.js Version
v18.16.1
Redis Server Version
Redis version=6.2.13
Node Redis Version
4.6.7
Platform
Ubuntu 22.04.2 LTS (WSL2)
Logs
[ErrorReply: Syntax error at offset 1 near >[]

The DIALECT should be directly in options, not in options.PARAMS (with your code sent to the server as a "user param" that is not in use...):
async function searchSimilar(vector: number[]) {
  const query = "*=>[KNN 10 @embedding $blob]";

  const options = {
    PARAMS: {
      blob: Buffer.from(new Float32Array(vector).buffer)
    },
    DIALECT: 2
  };

  const result = await client.ft.search("idx:answers", query, options);

  return result;
}

The DIALECT should be directly in options, not in options.PARAMS (with your code sent to the server as a "user param" that is not in use...):
async function searchSimilar(vector: number[]) {
  const query = "*=>[KNN 10 @embedding $blob]";

  const options = {
    PARAMS: {
      blob: Buffer.from(new Float32Array(vector).buffer)
    },
    DIALECT: 2
  };

  const result = await client.ft.search("idx:answers", query, options);

  return result;
}

Thanks, I just get tired with this one and didn't see a tree in a forest.

@DenisLutsky I'm not gonna lie, this command is a bit confusing, but it's not something we can "hide" on the client side...

@DenisLutsky I'm not gonna lie, this command is a bit confusing, but it's not something we can "hide" on the client side...

No, it's not you. I was inattentive and didn't see your examples. So I had to find the answers in different sources and in the end, I put DIALECT in wrong and section and I couldn't understand what I'm doing wrong


Uwp performance enhancement

This pull request appears to fix the issue: #429

So... the 1.4.x impl forces Skia to be refreshed everytime CompositionTarget.Rendering event is fired.  I don't know a lot about it under the covers, but here is the documentation.
The abridged version is that this event is not tied to any state of your app, or control, but to the XAML rendering engine itself (and I would guess it is more like a timer or simply as-fast-as-possible loop).  So, the MapControl is constantly being re-drawn, regardless of the fact that nothing may have changed.  You can verify by starting the UWP sample with debugger and get everything loaded, wait 10-20 seconds and then place a breakpoint or debug print on the CompositionTarget.Rendering event handler.  You'll see it is called constantly, even if no interaction with the app.  So, in my poking around, I simply removed that event handler and found that the map only updated @ the end of a drag.  The adding/removing of the CompositionTarget.Rendering event was to make sure that the map visibly updates during a drag (manipulation), but then doesn't re-draw if the map is static/idle.
As the comments point out, forcing Skia to re-draw from OnManipulationDelta event will also maintain map updates during a drag without using the  CompositionTarget.Rendering event at all, but in my admittedly limited testing, the impl in the pull request performs visually identically, with lower CPU/GPU usage.  Not sure why.  Maybe ManipulationDelta event fires more often than  CompositionTarget.Rendering event?  Maybe other code called from OnManipulationDelta() is being done more than once?

In the current master gpu is still high while the render method is not called. I don't understand.

In the new implementation the gpu is also high. And it is higher if you make the application window larger. The SKXamlCanvas has it's own render loop?

Normally (if I remember this right) Skia has two types of rendering methods: you could create a static image and let Skia draw this or you could use a function/event, that is called each time, when Skia canvas needs a redraw.

Compared them again. I barely see a difference. Perhaps the cpu is 2 instead 1.5. Do you see something else @jasells? Could it be your initial test was with an old version of uwp that did not check the _invalid boolean?

@charlenni Googling for skiasharp rendering loop did not give any answers. If you have some pointers where to look for I will try again.

From #429
Here is TaskManager screen-shot with current build (1.4).

Here is TaskManager screen-shot after fix.
Both are @ idle (no map interaction/changes happening). On my Surface Pro (i7 7660U, Intel Iris Plus 640 GPU)
My built in screen on the Surface is higher res than typical display (2736x1824), so likely is amplifying the issue somewhat, but the point is if no changes to the map are made, there is no reason to call SKXamlCanvas.Invalidate() and force a redraw.  The app sitting idle shouldn't be using any cycles CPU or GPU.
A breakpoint (or debug.Writeline) in `
private void CompositionTarget_Rendering(object sender, object e)
    {

        _renderTarget.Invalidate();

    }`

will hit (a lot), even when app is idle and nothing on screen changed.

@pauldendulk, your break point in above screen cap is in _renderTarget_PaintSurface() event handler.  These are different events.   Skia doesn't have a render loop, the app/runtime does.  I'm not sure exactly what triggers the PaintSurface event, but it is not being raised if nothing is changing in the app.  A quick test with a Debug.Writeline() shows that PaintSurface is called several times on start up, or if the window size changes, or zoom, etc, and then stops.

Oh, I see now that my test was not correct. Will test again tonight.

I pushed up another fix that originally thought was unrelated, but now realize it is.  Adding/Removing Layers wasn't visible on screen w/o some other trigger for a screen refresh.  Added a trigger in Mapsui.Map in LayersLayerRemoved/LayersLayerAdded to make sure the changes show on screen.  The reason is because the CompositionTarget.Rendering event was telling Skia to redraw @ ~60 HZ, so the changes got shown on a periodic basis (vs. event-driven).

I can reproduce your fix now. Looks great. Still not sure why there was such a high gpu because the render update was not actually called. The bind to CompositionTargetRendering trigger a render update somehow. In WPF a similar setup has no high gpu/cpu usage.
You now attach and detach to the render event. This was because of performance. I now removed the handler all together and call _renderTarget.Invalidate(). This seems to perform fine. What did you try?

btw, Refresh() updates both data and graphics.
        public void Refresh()
        {
            RefreshData();
            RefreshGraphics();
        }

-btw, Refresh() updates both data and graphics.
Yah, I didn't quite follow why that code was written that way.... most of the code seems to be event driven, but that appears to be more appropriate for a constant loop update (like from the CompositionTarget.Rendering event), or maybe from a input/touch event.

-You now attach and detach to the render event. This was because of performance. I now removed the handler all together and call _renderTarget.Invalidate(). This seems to perform fine. What did you try?

I don't quite follow without seeing your code...   Where are you calling Invalidate()?
Subscription model of events is very useful for this kind of dynamic sub/unsub, though I get that is not the most commonly seen usage of events in .NET.  Nothing inherently bad (nor inefficient) about doing it, but it can be a bit difficult to follow if done lots of places.  That's why I put OnManipulationStarted/Ended methods together in code.  Could also move CompositionTarget_Rendering closer to those to help too.

In WPF a similar setup has no high gpu/cpu usage
Not sure, I haven't looked @ that code much.  A quick look shows some extra logic to check for need to Invalidate() not in the UWP version... I did try something similar, but the _invalid flag in UWP code was used many places and I couldn't figure out how to duplicate it.  Slightly more efficient to just unsubscribe from the event vs. doing a if(flag) return check @ 60+ Hz.... most time is in un-sub state anyway. WPF version:

`
private void CompositionTargetRendering(object sender, EventArgs e)
    {

        if (!_invalid) return; // Don't render when nothing has changed

        if (RenderMode == RenderMode.Wpf) RenderWpf();

        else SkiaCanvas.InvalidateVisual();

    }`

About Refresh(). The Refresh(), RefreshData() and RefreshGraphics() methods are public methods intended for users. Perhaps they are used now on some points where we should use _renderTarget.Invalidate().
I suspect that there will be more places where we will have to subscribe and unsubscribe which will complicate the code. Calling _renderTarget.Invalidate() from OnManipulationDelta is somewhat simpler. I will accept your merge, and apply that change so you can take a look.
With the render loop totally removed it is perhaps also possible to remove the _invalid flag. Further simplifying the code.
What still confuses me is that in the old code the render method was called, but it exited right away, there was no actual rendering, yet the gpu was high. Somehow the skia view updated itself for some reason.

This is with the render loop removed and _invalid removed. In the manipulation delta method I call RefreshGraphics.
https://github.com/Mapsui/Mapsui/commit/45a027b5216203bdcd89f60bd9e42ad41fc2179d


[DARGA] Backport  1e70efc: Use reduced detail openstack availability zone API

Opened to solve VCR cassette merge conflicts detected while trying to backport https://github.com/ManageIQ/manageiq/pull/9672
@tzumainn @chessbyte

@mansam Can you change the commit message of the cherry-pick of the merge commit to what it actually contains...probably to Use reduced detail openstack availability zone API?  I only ask because it is going to be confusing in the history to see a non-merge commit with the commit message of "Merge Pull Request"

Yeah, absolutely.


https://material.angular.io/ down at the moment?

Visit chrome://serviceworker-internals/ and unregister the worker for material.angular.io. This will clear the service worker cache, which persists through hard refresh.
We've seen some issues where our hosting provider had a period of delivering truncated responses, causing a bad index to be cached.


#1649 Add a microbenchmark for QueryCounter

This PR adds a benchmark function for QueryCounter which includes a chain of QueryCounter followed by Loopback. It sends several redis commands to trigger the increment of the QueryCounter metric.
Progress towards #1649

for the regression on the encode_system.local_result_v5_no_compression benchmark, I think that changing the benchmarks has just added some noise by making the benchmark executable compile slightly differently or leave the heap in a slightly different state.
So dont worry about it.


Rasterizing Tile Layer V2 (Is a new Pull request with my changes)

I use in the Sample as Rendering Format Skp because this is what i intend to use by default.
The reason why the Scaling didn't work is that the NavigateTo Event was fired before the Animation was finished. So that not the correct Data where loaded. I introduced an Action to Animation Entry so that this Action is executed on the Final Animation and so solving the Problem.
In the comment of OnNavigated to I found the following command
// Note. Instead of a delay it may also be possible to call Navigated immediately with the viewport state
// that is the result of the animation.
But I don't know mapsui well enough to fix this.
@pauldendulk
So feel free to change this pull request if, I've done something not so good.

I have now fixed all Bugs that I know of.

@inforithmics Sorry I could not find time for this so far. My focus is now the NTS port and I will continue with that until I have a MR for the full port. I am happy with the progress there.

This pull request is not urgent and to have all planed features, I need NTS.

Sorry for causing conflicts. If you can fix them I will make time to coming week to review this.

@inforithmics It is really easy to use own style renderers. Much easier as to change Mapsui core parts ;-)
You could find an implementation in the vector tile layers (see #1478), where I use own style renderers for raster tiles, that have a special paint (could be transparent). See for this the RasterTileStyle and the corresponding RasterTileStyleRenderer. The style could be an empty class, that only should derive from IStyle. The renderer must implement ISkiaStyleRenderer.  Before you could use it, you have to register the renderer for the style
mapControl.Renderer.StyleRenderers[typeof(RasterTileStyle)] = new RasterTileStyleRenderer();

That's it. Now Mapsui always uses, if a feature or layer has this style, the given renderer by calling the
public bool Draw(SKCanvas canvas, IReadOnlyViewport viewport, ILayer layer, IFeature feature, IStyle style, ISymbolCache symbolCache)

function of the renderer.
At the end, all predefined renderer should use this pattern, so that you could easily replace a renderer by your own and have not to change Mapsui core functions. The only difference to the above would be, that the default renderers are set by Mapsui while the ones you define must be set by you.

I think I have to rearchitect things here. And I found cases where the Tiling Layer does not work. So again (WIP)

Thought about this a bit longer.
If it is right, what I wrote above, then you try to make a first-rendering-then-drawing process by a special layer. You convert all Mapsui geometries into Skia paths/commands, that then could be drawn on the screen. The advantage is, that you don't need to convert the geometries into Skia each time you want to draw them. The disadvantage is, that you don't have the information, which geometries should be drawn on the screen, because you don't know which part of the map is shown on the screen.
One solution would be to think about the whole map is a big canvas and use this to convert the geometries into Skia paths. When drawing, you calculate the origin and scale, move and scale the canvas and draw your already rendered geometries. But that would not work, because Skia use only floats for matrices and the error is on higher zoom levels to big.
Up to now, I don't have a good solution for this, because it should work with an unknown layer. Perhaps a solution with different zoom levels would work (create Skia paths/commands for each zoom level and different origins). Or calculate the matrices by yourself and transfer them later to Skia when you know something about the visible part of the map. Or anything else?

I fixed the Country Shape Tiling Smaple the reason it didn't work was different CRS because. Rasterizing Tiling Layer uses GlobalSphericalMercator and it now uses a IProjction to convert that.
Then I added a GeometrySimplyProvider (for simplifying Geometries) and an Sqlite PersistCache Sample. (I didn't inlcude it in Mapsui because it would add sqlite as a dependency).

@inforithmics Perhaps it is better to create a new PR with just the minimal changes to add tiled bitmap rasterization. The RasterizingTileSource should be the main big change, and apart from that I expect only a few changes. We can think about Skp later on.
In the meantime I going to reorganize the project structure, which will break things.

Agreed I will make V3 of this pull request with only rasterizing tile layer and png Format


absorb new meroxa-go interface; small doc changes

Description of change
absorb new meroxa-go interface; small doc changes
Fixes
Type of change

[ ]  New feature
[ ]  Bug fix
[ ]  Refactor
[ ]  Documentation

How was this tested?

[ ]  Unit Tests
[ ]  Tested in staging

Demo
Before this pull-request
Include how it looked before
After this pull-request
Include how it looks now
Additional references
Any additional links (if appropriate)
Documentation updated
Make sure that our documentation is accordingly updated when necessary.
You can do that by opening a pull-request to our (🔒 private, for now) repository: https://github.com/meroxa/meroxa-docs.
✨ In the future, there will be a GitHub action taking care of these updates automatically. ✨
Provide PR link:

@janelletavares started reviewing this pull-request, but I think it lacks some context on the motivation behind it. Maybe adding a link to the change made in meroxa-go would be useful. I appreciate we're moving away from having the CLI to define their own mocks, but I noticed some other UX changes (messaging, etc) that would be great to detail.
Let me know if I can help with this.

@janelletavares started reviewing this pull-request, but I think it lacks some context on the motivation behind it. Maybe adding a link to the change made in meroxa-go would be useful. I appreciate we're moving away from having the CLI to define their own mocks, but I noticed some other UX changes (messaging, etc) that would be great to detail.
Let me know if I can help with this.

Here are the related PRs, if that helps.
https://github.com/meroxa/meroxa-go/pull/77
https://github.com/meroxa/meroxa-go/pull/78
https://github.com/meroxa/meroxa-go/pull/79

@janelletavares Working on the CLI today I realized I'm blocked on some changes made in meroxa-go, so I'll need make some of these changes myself to unblock environments' work, unless this pull-request is broken out in different parts. My biggest concern on this pull-request are the user breaking changes that would need to be communicated to customers (removing support on feature flags).
If we could have only one pull-request that could make the CLI work with the latest of meroxa-go would be great, thank you.

@raulb if the meroxa-go PR and this PR seem good to you, feel free to squash merge, update the go.mod to unblock you.


Strange issue related to the use of substring in ParserInput

I discovered a strange issue that seems to be linked to the use of substring in ParserInput.
My apologies, I wasn't able to create a minimal example. But the issue can be observed by running the tests of my LogicKit library, with the latest changes merged in master. At line 44 of Parser.swift, I parse conjunction of terms, using infixedLeft(by:). Everything works fine with two terms, but if I attempt to parse three, the parser produces completely invalid results (In that particular example, an invalid value of the enum) that may even trigger runtime errors.
Using a String rather than a Substring for the property ParserInput.stream seems to solve the problem. Therefore, my guess is that something wrong is happening with the pointers of the substring when the parser backtracks. Unfortunately I don't have any evidence to support that claim, and I don't know how the current implementation differs from the previous one (which was based on character views).
Any idea on the source of the problem?
Also, I understand some concerns were raised about the use of strings in ParserInput, as they may involve a lot of unnecessary copies. But maybe there is an intermediate solution that could rely on Swift's default copy-on-write behavior since strings are value-types? Maybe some benchmarks could shed some light on the issue.

It's highly unlikely that it's a problem with the usage of Substring. What's your string input when you are parsing two terms? And what is it when you are parsing three?


[Bug]: Transaction redirects from Buy Airtime to Check Balance When Buying Airtime with Equity bank to Safaricom

Description
After inputing your Equity Bank pin inorder to buy airtime with Equity Bank to Safaricom, the Checking Equity Bank Balance (communicating with USSD service) is the next screen. This then redirects to Check Balance Transaction Status page which shows the transaction status as failed and transaction type as Check balance
Steps to reproduce

Open Stax app.
On the home screen click on the Buy airtime icon.
On the pay with input field, select Equity bank.
On the recipient network input field, select Safaricom.
Next , on the Contact phone number input field, enter your phone number
On the “Amount” input field, enter the amount above 10ksh and click continue.
When prompted with the “Finish importing account” modal click connect now.
Pay attention to the transaction details page.

Expected result
The next screen should be Buy Airtime with MPESA screen(communicating with USSD service) which will then redirect to the Buy Airtime Transaction status which will show transaction type as Buy AirTime
Actual result
The next screen is Check Balance screen(communicating with USSD service) which will then redirects to Check Balance Transaction Status page which shows the transaction status as failed and transaction type as Check balance
Link to screenshots/screen recordings
https://drive.google.com/file/d/1NuAsVevqAn7QJwx9UCMdKhu-tVNrmQDp/view?usp=sharing
Device name
Samsung A22
Android OS
12
App version
1.17.3
Config version
20221109082804
Is your phone dual SIM or single SIM?
Dual SIM
Which telco are you using for this test?
Safaricom
Stax device ID or email address
<EMAIL_ADDRESS>

@Sally-N it looks like your account list is messed up due to issues in our previous app version/equity configuration. Could you please uninstall or clear data and then re-install?

@davkutalek After clearing cache, my transactions are failing. I'm gonna uninstall then re-install

@davkutalek after re-installing check balance, send money and buy airtime are still failing


Update .travis.yml

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@6c6d993). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master      #3   +/-   ##
========================================
  Coverage          ?   94.8%
========================================
  Files             ?       2
  Lines             ?     250
  Branches          ?       0
========================================
  Hits              ?     237
  Misses            ?      13
  Partials          ?       0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c6d993...8001fab. Read the comment docs.


Filed resolve dependencies

When i try add this line: implementation 'com.github.florent37:runtime-permission-kotlin:1.0.7'
in my graddle configuration file i get this error: Failed to resolve: org.jetbrains.kotlin:kotlin-stdlib-common:1.3.0-rc-131. For version 1.0.0 everything works right.

I updated it to 1.3.0

Add the last version info to the Readme


Inconsistency in F1 metric between manual eval and Trainer.test() run

🐛 Bug
When training a multilabel image classifier as described in the docs,
(original link:https://lightning-flash.readthedocs.io/en/latest/reference/multi_label_classification.html),
import os.path as osp
from typing import List, Tuple

import pandas as pd
from torchmetrics import F1

import flash
from flash.core.classification import Labels
from flash.core.data.utils import download_data
from flash.image import ImageClassificationData, ImageClassifier
from flash.image.classification.data import ImageClassificationPreprocess

# 1. Download the data
# This is a subset of the movie poster genre prediction data set from the paper
# “Movie Genre Classification based on Poster Images with Deep Neural Networks” by Wei-Ta Chu and Hung-Jui Guo.
# Please consider citing their paper if you use it. More here: https://www.cs.ccu.edu.tw/~wtchu/projects/MoviePoster/
download_data("https://pl-flash-data.s3.amazonaws.com/movie_posters.zip", "data/")

# 2. Load the data
genres = ["Action", "Romance", "Crime", "Thriller", "Adventure"]

def load_data(data: str, root: str = 'data/movie_posters') -> Tuple[List[str], List[List[int]]]:
    metadata = pd.read_csv(osp.join(root, data, "metadata.csv"))
    return ([osp.join(root, data, row['Id'] + ".jpg") for _, row in metadata.iterrows()],
            [[int(row[genre]) for genre in genres] for _, row in metadata.iterrows()])

train_files, train_targets = load_data('train')
test_files, test_targets = load_data('test')

datamodule = ImageClassificationData.from_files(
    train_files=train_files,
    train_targets=train_targets,
    test_files=test_files,
    test_targets=test_targets,
    val_split=0.1,  # Use 10 % of the train dataset to generate validation one.
    image_size=(128, 128),
)

# 3. Build the model
model = ImageClassifier(
    backbone="resnet18",
    num_classes=len(genres),
    multi_label=True,
    metrics=F1(num_classes=len(genres)),
)

# 4. Create the trainer. Train on 2 gpus for 10 epochs.
trainer = flash.Trainer(max_epochs=10)

# 5. Train the model
trainer.finetune(model, datamodule=datamodule, strategy="freeze")

# 6. Predict what's on a few images!
# Serialize predictions as labels, low threshold to see more predictions.
model.serializer = Labels(genres, multi_label=True, threshold=0.25)

predictions = model.predict([
    "data/movie_posters/predict/tt0085318.jpg",
    "data/movie_posters/predict/tt0089461.jpg",
    "data/movie_posters/predict/tt0097179.jpg",
])

print(predictions)

# 7. Save it!
trainer.save_checkpoint("image_classification_multi_label_model.pt")

I get different F1 metrics for the test set depending on how I run the evaluation:
# Run test with trainer:

trainer.test(model, datamodule=datamodule)

# stdout:
# {'test_binary_cross_entropy_with_logits': 0.5449734330177307,
# 'test_f1': 0.46086955070495605}

# Run test manually:

metric = F1(num_classes=len(genres))

for batch in datamodule.test_dataloader():
    image_tensor = batch[DefaultDataKeys.INPUT]
    target = batch[DefaultDataKeys.TARGET]
    with torch.no_grad():
        y_hat = model(image_tensor)
    prediction = model.to_metrics_format(y_hat)
    metric(prediction, target)

print(metric.compute())

# stdout:
# tensor(0.3891)

To Reproduce
Steps to reproduce the behavior:

Copy paste the example training code from the link above
Add the test evaluation code above
Save and run the script
See error

Expected behavior
The two F1 metrics should be identical
Environment

PyTorch Version: 1.8.0
PyTorch-Lightning: 1.3.5
Lightning-Flash: 0.3.2
Torchmetrics: 0.3.2
OS (e.g., Linux): macOS
How you installed PyTorch (conda, pip, source): pip
Python version: 3.8.8
CUDA/cuDNN version: N/A
GPU models and configuration: None
Any other relevant information: None

Additional context
None

Hi @lillekemiker,
Finally figured out what the issue is. So the difference lies in that a default transform is applied to datamodule.test_dataloader() when calling the trainer.test method (in the case of classification it is a standard normalization: https://github.com/PyTorchLightning/lightning-flash/blob/da684414f09cac8a65d412814491124343c8b416/flash/image/classification/transforms.py#L60) but when calling datamodule.test_dataloader() outside the trainer object it is not.
@tchaton is this the expected behaviour.

Thank you!!
This has been driving me crazy.
Don't know if it is expected behavior but considering that transforms are defined on the datamodule level, I definitely expected them to be applied. How would I even turn them on outside the trainer?

Hi @SkafteNicki @lillekemiker - this is definitely something we could do better. The challenge is that some of our transforms are applied in the dataloader and some are applied in the model. So at runtime we inject the transforms into the dataloader / model. There's a few options:

We could provide a public facing API for doing this injection. Something that takes a model and a dataloader and injects the transforms correctly.
We could just inject the dataloader transforms (pre_tensor_transform -> ... -> per_batch_transform) and maybe warn the user that any transforms that would usually be injected into the model won't be applied
We could just not apply any transforms but warn the user that none of the transforms they provide will be injected into the dataloader
Some combination of warning / injecting

Interested to hear your thoughts :smiley:

What is the argument against having the dataloader handle all transforms without magic runtime injections?

@lillekemiker because we support transforms on device. So if the user provides per_sample_transform_on_device or per_batch_transform_on_device they have to be injected into the model rather than the dataloader.

I can see the logic behind this design decision. I don't agree with it, though, I don't think.
As an abstraction, the dataloader should handle transforms. Technically, the dataloader often runs in multiple threads per GPU and so because you don't want multiple threads using the same GPU, you moved the GPU transforms into the model realm. Or at least I assume this is the reasoning. Am I missing anything?
I think as a design decision, it is more important to keep the dataloader/model contact surface clean. Couldn't GPU transforms be applied in the same thread as the model before handing off the batch to the model rather than after?
A GPU threadlock is another option but that might come with a performance penalty.
As such, transforms could live in the model, too, and simply be part of the model. I don't see anything conceptually wrong with that.

One major issue with the current way of doing it is that depending on whether or not Kornia is installed, the tensor normalization may be happening in per_batch_transform_on_device or in post_tensor_transform and so may or may not have happened in the manual case without a Trainer.

IMO if this was barebone lightning then I would expect that a batch looked the same regardless if it is inside the lightning trainer or outside, because lightning is "just reorganised pytorch code".
However, if this also should be the case in flash I am not completely sure about. It depends completely on the design philosophy of flash. Since it is at an higher abstraction than lightning, I am fine with this not being supported.
@ethanwharris is it possible to extract the data pipeline such that it would be possible to do something like
model(pipeline(batch))

I may also be a bit too unclear about Flash's design philosophy. Either way, though, if the dataloader is exposed to the end user (i.e. me), then I would expect it to either always apply all transforms or never apply any transforms, and that it would be consistent in its behavior. If the dataloader is an internal and I'm never supposed to see it, then I guess I shouldn't really care :)
As for the design philosophy, my personal vote would be that flash should be highly modular and made in a way making it easy to dismantle and replace parts of it with custom code. That way it provides a quick and easy baseline model with very little coding, but more importantly, once you need to move beyond the baseline model, you don't have to start over and rewrite the whole thing in lightning yourself. You just replace the parts that need replacing.
If that is not what you guys are going for, though, that is also completely fair. Blackbox, off-the-shelf deep learning solutions definitely have their place too.

@SkafteNicki that could be possible. I think the issue with what we have now is that we inject the transforms into the correct places inside the trainer. What we could definitely do is expose a method like:
dataloader = datamodule.train_dataloader()
model, dataloader = datamodule.inject_transforms(model, dataloader)

If we documented that as the recommended way to use the flash datamodules without a lightning Trainer, I guess that would address some of the issues here? @tchaton Interested to hear your thoughts


Cmd+W should close a current tab

What version of Remix are you using?
Desktop app
Version 1.3.4 (<IP_ADDRESS>5)
macOS 12.0.1 (21A559)
Steps to Reproduce

Open Remix
Open a file
Press Cmd+W

Expected Behavior
Close the current file tab, as most (all?) other IDEs do.
Actual Behavior
Closes the whole app.

Hello 👋
I think you may be in the wrong place. This GitHub repo is all about the Remix Web Framework which you can learn about at https://remix.run.
Closing this as you have mixed this up with the Remix IDE (https://remix-project.org) which is a completely different project.