CLOUDSTACK-9542 make listNics and ListUserVms return uniform NIC data

@PaulAngus @borisroman please see if this is what is expected

Hey @DaanHoogland I think you pinged the wrong Boris, but don't worry
@blueorangutan package

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

sorry @borisroman , just missing you ;)

Packaging result: ✔centos6 ✔centos7 ✔debian.  @DaanHoogland however a minor note, the comments may be indented, or just removed. I understand that it notes the sequence of keys, however that's not how the params/keys are serialized/returned in the api response. It just makes the code look bit weird IMO :)

@rhtyd I agree with your comment on comments (in this case) however those are there just to keep track for the viewer which has been added in which output. I will add cross references to both bits to each other so future generations will be able to make sense of it.

@borisstoyanov I don't see how a host unreachable can be caused by an API result changing. I will rebase, repackage and retest.

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos6 ✖centos7 ✖debian. JID-1021

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1028

ACS CI BVT Run
Sumarry:
Build Number 1146
Hypervisor xenserver
NetworkType Advanced
Passed=106
Failed=5
Skipped=12
Link to logs Folder (search by build_no): https://www.dropbox.com/sh/r2si930m8xxzavs/AAAzNrnoF1fC3auFrvsKo_8-a?dl=0
Failed tests:

test_volumes.py

test_06_download_detached_volume Failing since 2 runs

test_routers_network_ops.py

test_01_isolate_network_FW_PF_default_routes_egress_true Failing since 11 runs

test_02_isolate_network_FW_PF_default_routes_egress_false Failing since 138 runs

test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true Failing since 133 runs

test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failing since 133 runs

Skipped tests:
test_vm_nic_adapter_vmxnet3
test_01_verify_libvirt
test_02_verify_libvirt_after_restart
test_03_verify_libvirt_attach_disk
test_04_verify_guest_lspci
test_05_change_vm_ostype_restart
test_06_verify_guest_lspci_again
test_static_role_account_acls
test_11_ss_nfs_version_on_ssvm
test_nested_virtualization_vmware
test_3d_gpu_support
test_deploy_vgpu_enabled_vm
Passed test suits:
test_deploy_vm_with_userdata.py
test_affinity_groups_projects.py
test_portable_publicip.py
test_vm_snapshots.py
test_over_provisioning.py
test_global_settings.py
test_router_dnsservice.py
test_scale_vm.py
test_service_offerings.py
test_routers_iptables_default_policy.py
test_loadbalance.py
test_routers.py
test_reset_vm_on_reboot.py
test_deploy_vms_with_varied_deploymentplanners.py
test_network.py
test_router_dns.py
test_non_contigiousvlan.py
test_login.py
test_deploy_vm_iso.py
test_list_ids_parameter.py
test_public_ip_range.py
test_multipleips_per_nic.py
test_metrics_api.py
test_regions.py
test_affinity_groups.py
test_network_acl.py
test_pvlan.py
test_nic.py
test_deploy_vm_root_resize.py
test_resource_detail.py
test_secondary_storage.py
test_disk_offerings.py

@blueorangutan  test

question about @cloudmonger, everyone: How to find the error output for the failed tests in the dropbox link. There are only numbered directories in there. Must they be grep'd, i.e. all downloaded to search? or can i find a logical link from a failed test name to a location of it's results? @karuturi @kishankavala @sanjaytripathi , @all, @anyoneelse, ...

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1544)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40738 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2208-t1544-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_iso.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 53 look OK, 4 have error(s)

Test
Result
Time (s)
Test File

test_01_vpc_remote_access_vpn
Failure
66.28
test_vpc_vpn.py

test_04_rvpc_privategw_static_routes
Failure
481.94
test_privategw_acl.py

test_05_iso_permissions
Failure
0.06
test_iso.py

test_02_edit_iso
Failure
0.06
test_iso.py

ContextSuite context=TestSnapshotRootDisk>:teardown
Error
97.37
test_snapshots.py

test_change_service_offering_for_vm_with_snapshots
Skipped
0.00
test_vm_snapshots.py

test_09_copy_delete_template
Skipped
0.02
test_templates.py

test_06_copy_template
Skipped
0.00
test_templates.py

test_static_role_account_acls
Skipped
0.02
test_staticroles.py

test_11_ss_nfs_version_on_ssvm
Skipped
0.02
test_ssvm.py

test_01_scale_vm
Skipped
0.00
test_scale_vm.py

test_01_primary_storage_iscsi
Skipped
0.04
test_primary_storage.py

test_vm_nic_adapter_vmxnet3
Skipped
0.00
test_nic_adapter_type.py

test_nested_virtualization_vmware
Skipped
0.00
test_nested_virtualization.py

test_06_copy_iso
Skipped
0.00
test_iso.py

test_deploy_vgpu_enabled_vm
Skipped
0.03
test_deploy_vgpu_enabled_vm.py

test_3d_gpu_support
Skipped
0.04
test_deploy_vgpu_enabled_vm.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1122

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1550)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 2393 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2208-t1550-kvm-centos7.zip
Test completed. 0 look OK, 1 have error(s)

Test
Result
Time (s)
Test File

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian.  I don't see direct relation to the code but don't dare to merge. I will rebase and go from there.

@DaanHoogland please rebase and fix conflicts

@DaanHoogland can you fix conflicts on your PR?

Yes @rhtyd I will rebase one last time.

@DaanHoogland thanks, you know you can kick tests yourself!
@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1438

@DaanHoogland and also run them ;)
@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1841)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30267 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2208-t1841-kvm-centos7.zip
Smoke tests completed. 64 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_vpc_privategw_acl
Failure
46.77
test_privategw_acl.py

test_02_vpc_privategw_static_routes
Failure
148.00
test_privategw_acl.py

test_03_vpc_privategw_restart_vpc_cleanup
Failure
193.41
test_privategw_acl.py

test_04_rvpc_privategw_static_routes
Failure
304.25
test_privategw_acl.py

test_02_create_template_with_checksum_sha1
Error
5.24
test_templates.py

test_03_create_template_with_checksum_sha256
Error
5.23
test_templates.py

test_04_create_template_with_checksum_md5
Error
5.24
test_templates.py

test_01_vpc_remote_access_vpn
Error
45.86
test_vpc_vpn.py

Tests lgtm, additional review required for merging this.

LGTM, based on the latests test results

Merged this based on test results and two lgtms/code reviews.


Use single database query in do_incremental_logging_stat

We use an inefficient database query construction for do_incremental_logging_stat:
    row, created = table._default_manager.get_or_create(
        property=stat.property,
        subgroup=subgroup,
        end_time=end_time,
        defaults={"value": increment},
        **id_args,
    )
    if not created:
        row.value = F("value") + increment
        row.save(update_fields=["value"])

Django (and postgres) support doing this operation in a single SQL query, perhaps using https://docs.djangoproject.com/en/5.0/ref/models/expressions/#f-expressions. I'm not sure whether https://docs.djangoproject.com/en/5.0/ref/models/querysets/#update-or-create will do the right thing in Django 5, or if it'll do multiple queries.
You can use this settings tweak:
diff --git a/zproject/computed_settings.py b/zproject/computed_settings.py
index 8d691753da..813a896956 100644
--- a/zproject/computed_settings.py
+++ b/zproject/computed_settings.py
@@ -864,11 +864,11 @@ LOGGING: Dict[str, Any] = {
             "propagate": False,
         },
         ## Uncomment the following to get all database queries logged to the console
-        # 'django.db': {
-        #     'level': 'DEBUG',
-        #     'handlers': ['console'],
-        #     'propagate': False,
-        # },
+        "django.db": {
+            "level": "DEBUG",
+            "handlers": ["console"],
+            "propagate": False,
+        },
         # other libraries, alphabetized
         "django_auth_ldap": {
             "level": "DEBUG",

to make Django print all database queries to verify your work.
See this chat.zulip.org thread for details.

@zulipbot claim

Can I work on this issue?

@zulipbot claim


Merge release 0 13 0 rc0

requires https://github.com/cdapio/hydrator-plugins/pull/978 to be merge first


implement buffer deadline to support websockets

This is just an experiment using channel locks, closing in favor of #98


exec /startup.sh: no such file or directory

When attempting to run docker compose up, I encountered an error related to the execution of a startup script (startup.sh). The error message states: "exec /startup.sh: no such file or directory". This issue occurred after successfully building the Docker image and creating the necessary network and container.
Steps to Reproduce:
Clone the repository or navigate to the directory containing the docker-compose.yml file.
Run docker compose up.
Expected Behavior:
The Docker container should start successfully and execute the startup.sh script as part of its initialization process.
Actual Behavior:
The following error is encountered:
win10-1  | exec /startup.sh: no such file or directory
win10-1 exited with code 1

Additional Information:
The startup.sh script is included in the root directory of the Docker context and is correctly copied to the root directory (/) of the container during the build process.
The permissions for the startup.sh script are set correctly to allow execution within the container.
Manually accessing the container and verifying the presence and permissions of the startup.sh script did not reveal any issues.
´´´
docker compose up
[+] Building 4.4s (18/18) FINISHED                                                                                docker:default
[win10 internal] load build definition from Dockerfile                                                                                0.0s
transferring dockerfile: 1.09kB                                                                                0.0s
[win10] resolve image config for docker.io/docker/dockerfile:1.5                                                                                1.3s
[win10 auth] docker/dockerfile:pull token for registry-1.docker.io                                                                                0.0s
CACHED [win10] docker-image://docker.io/docker/dockerfile:1.5@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14                0.0s
[win10 internal] load .dockerignore                                                                                0.0s
transferring context: 2B                                                                                0.0s
[win10 internal] load metadata for docker.io/library/ubuntu:22.04                                                                                2.7s
[win10 auth] library/ubuntu:pull token for registry-1.docker.io                                                                                0.0s
[win10 1/9] FROM docker.io/library/ubuntu:22.04@sha256:e9569c25505f33ff72e88b2990887c9dcf230f23259da296eb814fc2b41af999                              0.0s
[win10 internal] load build context                                                                                0.0s
transferring context: 63B                                                                                0.0s
CACHED [win10 2/9] RUN apt-get update -y &&     apt-get install -y     qemu-kvm     build-essential     libvirt-daemon-system     libvirt-dev     o  0.0s
CACHED [win10 3/9] RUN wget https://releases.hashicorp.com/vagrant/2.4.0/vagrant_2.4.0-1_amd64.deb &&     apt install ./vagrant_2.4.0-1_amd64.deb &  0.0s
CACHED [win10 4/9] RUN vagrant plugin install vagrant-libvirt                                                                                0.0s
CACHED [win10 5/9] RUN vagrant box add --provider libvirt peru/windows-server-2022-standard-x64-eval &&     vagrant init peru/windows-server-2022-s  0.0s
CACHED [win10 6/9] COPY Vagrantfile /Vagrantfile.tmp                                                                                0.0s
CACHED [win10 7/9] RUN rm -rf /Vagrantfile                                                                                0.0s
CACHED [win10 8/9] COPY startup.sh /                                                                                0.0s
CACHED [win10 9/9] RUN chmod +x startup.sh                                                                                0.0s
[win10] exporting to image                                                                                0.0s
exporting layers                                                                                0.0s
writing image sha256:743b19d51465b5dddba673dbab2c0e1154c2522dc37a63c747f0da8f30f1d181                                                             0.0s
naming to docker.io/library/dockervm-win10                                                                                0.0s
[+] Running 2/2
✔ Network dockervm_default    Created                                                                                0.1s
✔ Container dockervm-win10-1  Created                                                                                0.1s
Attaching to win10-1
win10-1  | exec /startup.sh: no such file or directory
win10-1 exited with code 1
´´´

Hi @kelcalegari,
Thanks for bringing this to our attention and for your valuable feedback.
I attempted to recreate the issue you're experiencing, but without success. To better help you, may I please have the following information:

Your Docker version
Your Operating System

It could be possible that the issue arose from local modifications made prior to building. If you've made changes in your local environment, I would recommend taking the following steps:
Try to rebuild and run the image using the following command: docker compose up --build
Alternatively, you can use the following image directly to circumvent the issue:
ghcr.io/vaggeliskls/windows-in-docker-container:latest

For usage, please refer to the example provided in the Readme.md file

This will hopefully help us diagnose the issue better. Looking forward to your response.
Best Regards


Problem after updating to home assistantt 2024.11

On my devices with Chrome, after updating to the new version of Home Assistant, the event videos are no longer loading.
In the Home Assistant app it works normally, the problem is on all devices that run the home assistant in chrome

Seems to be a common issue with the latest Home Assistant Versiom

Duplicate: https://github.com/dermotduffy/frigate-hass-card/issues/1678


Dropdown Hover:true not working while hover menu

when i set hover:true in dropdown menu, while hover link its appear but i can't select the dropdown links it will disappear, any solution for this ?..

@cvaraj
Materialize.js
Line 463
Replace
  hover: false,  with
hover: true
The dropdown will work and select even on hover.

thank you @vishnu1991 , its workinng . but when i use out of Materialize.js . like my footer or custom.js its not working ..

you can change the respective minified file(materialize.min.js) same as the materialize.js file.

ok @vishnu1991 i will check .. Thank you

The recommended solution to this bug is for us to modify your code? Lol. That seems a bit absurd. This bug should be re-opened, and actually fixed.


Update dspbr-pt 0.1.0 -> 0.2.0

Changes

Updates dspbr-pt renderer to the latest version
Renaming the viewer to match the renderer npm package name
The renderer update enables support for more scenarios

Relevant renderer feature updates and fixes:

Adds support for glTF alpha blending modes
Adds feature to disable background rendering (->transparent background)
Path termination logic is now aware of specular events and keeps on bouncing. This removes the need for high bounce depths in scenarios with lots of specular and/or transparent objects.
Fixes a numerical issue with path sampling probabilities which in certain scenarios caused image darkening/contrast increase and excessive noise.
Enables ACESFilmic tonemapping by default to match the model-viewer behavior

Thanks, I'm really looking forward to seeing the new renders! Can you please commit the new pt goldens? You might also have the change the names of the existing goldens to match your name change. The CI fidelity tests are failing because they can't find the golden files.

@elalish Goldens are still on progress. I could just manage to get the fidelity-tools package to build and run on my windows machine, so I can now make use of a GTX1060 to render the images. Looks like I'm able to render the images in full resolution with a proper amount of samples in a reasonable amount of time...

@bsdorra Spectacular! I don't have access to a Windows machine and I know we've gotten reports of difficulty in getting a windows dev environment set up for model-viewer. If you notice anything that would make it work better, contributions would be most welcome!

@elalish well, bash commands and symlink usage prevent native windows bootstrapping and building. From WSL it is pretty straight forward. I had to manually copy over the shared-assets folder to render-fildelity-tools, it is only symlinked otherwise. To run update-screenshots, I also had to reinstall the puppeteer package from windows cmd to get the chromium win binaries. That's pretty much it. Hacky, but it seems to work :)

@bsdorra Thanks, super helpful to know it works through WSL. Did you try building all the packages (npm run build from the root)? If that works along with npm run test then I'd declare it a fully operational dev environment. Would you be willing to write up some steps I could put on our website to help people get up and running on Windows? And is there anything we could change in our repo that would make it easier?

@bsdorra yes, I was able to run npm run build from root, I didn't run test though.
Let me try to reproduce what I did on a fresh clone. I'll document the process and try to think about potential changes to make the process easier.


Update Innovation badge for 2018 and css with Web Standards fixes

Update Innovation badge with 2018 version
Little footer bar height must be 59px
Little footer links should not have any underline decorations, nor change color on hover/active
Little footer text font size is to be 16px
Innovation text size is supposed to 24px
Adjust padding for text in little footer to ensure center vertical alignment
Innovation bar height must be 56px
Links in Innovation bar must not change color on hover/active
Adjust Innovation image margins for better placement

Seems to make sense to me.


Support PostgreSQL exponentiation.

As of current version, SELECT 3 ^ 4 in the PostgreSQL dialect reads as BinaryOperator::BitwiseXor, while it should read as the exponentiation operation. I added BinaryOperator::PGExp for that purpose and completed all the tests in this pull request.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

17 of 17 (100.0%)  changed or added relevant lines in 5 files are covered.
378 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.008%) to 86.148%

Files with Coverage Reduction
New Missed Lines
%

tests/sqlparser_common.rs
1
97.32%

src/parser.rs
377
83.23%

Totals

Change from base Build<PHONE_NUMBER>:
0.008%

Covered Lines:
13409

Relevant Lines:
15565

💛  - Coveralls


TypeError: The input to be decoded is not correctly encoded - when upgrading from 0.7.6 to 0.8.6

I was trying to upgrade the library version from 0.7.6 to the latest one 0.8.6 and none of the currently logged-in users could use the app.
For all users logged in with the previous version (0.7.6), the app started to refresh every second, not letting them interact with it.
Looking at the logs on the server, I see the following:
[GET] [middleware: "middleware"] /api/login reason=EDGE_FUNCTION_INVOCATION_FAILED, status=500, user_error=true

and
TypeError: The input to be decoded is not correctly encoded.
    at (node_modules/jose/dist/browser/runtime/base64url.js:35:14)
    at (node_modules/next-firebase-auth-edge/lib/auth/rotating-credential.js:12:35)
    at (node_modules/next-firebase-auth-edge/lib/auth/rotating-credential.js:16:20)
    at (node_modules/next-firebase-auth-edge/lib/auth/cookies/sign.js:11:38)
    at (node_modules/next-firebase-auth-edge/lib/next/cookies.js:34:98)
    at (node_modules/next-firebase-auth-edge/lib/next/cookies.js:59:11)
    at (node_modules/next/dist/esm/server/web/adapter.js:145:19)

Seems like the app is not able to properly get or set the user from the cookies.
Have I done something wrong? Is there any way to prevent this from happening?
Thanks a lot for the help!

After making several changes and running the app locally (both prod build and dev build), I got the following errors:
- error node_modules/next-firebase-auth-edge/lib/auth/token-verifier.js (80:15) @ FirebaseTokenVerifier.mapJoseErrorToAuthError
- error Internal error: fetch failed

Error: Internal error: fetch failed
at FirebaseTokenVerifier.mapJoseErrorToAuthError (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/token-verifier.js:79:16)
at eval (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/token-verifier.js:65:24)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async FirebaseTokenVerifier.decodeAndVerify (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/token-verifier.js:33:9)

at async FirebaseTokenVerifier.verifyJWT (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/token-verifier.js:21:25)
at async verifyIdToken (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/index.js:130:32)
at async getCustomIdAndRefreshTokens (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/auth/index.js:156:24)
at async setAuthCookies (webpack-internal:///(middleware)/./node_modules/next-firebase-auth-edge/lib/next/cookies.js:54:32)
at async adapter (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/web/adapter.js:161:20)
at async runWithTaggedErrors (/node_modules/next/dist/server/web/sandbox/sandbox.js:98:24)

Hey @MartinXPN, thanks for reporting :-)
Could you run npx next info in your project and paste the result here? It should give me some context on Node and Next versions you're using. Also, could you share a list of firebase sdk packages your app depends on?

Sure, here is the result of npx next info:
    Operating System:
      Platform: darwin
      Arch: x64
      Version: Darwin Kernel Version 21.6.0: Thu Jul  6 22:18:26 PDT 2023; root:xnu-8<IP_ADDRESS>2.13~1/RELEASE_X86_64
    Binaries:
      Node: 18.17.0
      npm: 9.6.7
      Yarn: 1.22.19
      pnpm: N/A
    Relevant Packages:
      next: 13.4.19
      eslint-config-next: 13.4.19
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.2.2
    Next.js Config:
      output: N/A

I'm currently using several firebase-related packages:
    "firebase": "^10.4.0",
    "firebase-admin": "^11.10.1",
    "firebaseui": "^6.1.0",
    "next-firebase-auth-edge": "^0.8.6",

I cannot reproduce the issue with those specific versions on https://next-firebase-auth-edge-starter.vercel.app/, but I have a feeling it's connected with the value of cookieSignatureKeys. Not 100% sure though.
Could you install<EMAIL_ADDRESS>and check if the error is still happening?

If it does still happen, will it start working after clearing browser cache?

Yeah, I just tried the<EMAIL_ADDRESS>version and it worked!
Thank you so much!
May I ask what was the issue?

Here's the PR with the fix https://github.com/awinogrodzki/next-firebase-auth-edge/pull/93/files
Before 0.8.7-1, I was using base64url.decode from jose  to convert cookie signature key to UInt8Array to generate cookie signature.
I think cookieSignatureKeys you pass to authentication middleware function have some unusual character or set of characters that cannot be decoded by base64url.decode function. It was mistake on my side. I tested base64url.decode against number of combinations and assumed it was safe for any string, but the assumption was false.
Anyways, converting to UInt8Array directly is much more efficient in this case. I'm glad the fix have worked!

Yeah, works like a charm. Thank you so much for fixing it this quickly!

Fix released in 0.8.7


@orta -> changes for deploying beta

:+1:

I don’t understand why this had to change again, so we definitely need to sit down together and figure out the differences in profiles etc and settle on 1 set. Added to https://github.com/artsy/mobile/issues/22.


nothing happend when i tap cicle floating button

my app just hangging when i tap the circle floating button, i build my app with swiftui with ios 16 minimum support

Could you provide more info:
Xcode Version, if possible create a repo demo and comment the repo url.

Okey will to it later today, thanks for replying bro

i know the triggers, it should be when the apps hit more networking, like open an gmaps view

Hello @imrobbyrc!
Could you please share more information about other dependencies.
Are you doing some network request initially? Which version of DebugSwift are you using?
Thanks,
Emil


OpenHere: Replace the fxplorer Window lookup code with site lookup

When we first introduced the shell extension, it didn't work properly for some folders (such as the Desktop, or perhaps any "background" click) due to a bug in Windows. We worked around that bug with the help of an awesome community member, who contributed code that would pull up the topmost Explorer window and query its location.
That Windows bug was eventually fixed, but we still had trouble with items appearing correctly. On Windows 11, the Open in Terminal menu item appears and disappears at random when you right-click the desktop, but it always appears when you right-click a folder. It sometimes appears for Quick Access, even though it shouldn't.
We tried to fix that in #13206, but the fix caused more issues than it solved. We reverted it for 1.15 and 1.16.
At the end of the day, it turns out that getting the path from the toplevel explorer window is fragile. Fortunately, the shell does offer us a way to get that information: the site chain.
This pull request replaces GetPathFromExplorer() with an implementation of IObjectWithSite, which allows us to use the site chain to look up from whence a context menu request was initiated. It also makes item lookup generally more robust.

✅ Tested on Windows 11

✅ Desktop
✅ Folder Background
✅ Folder Selected
✅ Quick Access (does not appear)
✅ This PC (does not appear)

References #13206
References #13523
Closes #12578

/cc @leejy12 :) thanks for looking at this with us! I'd appreciate your review and signoff, if you can, since you're the most familiar with this code.

I have to test this on Windows 10 19041!

I have updated to Terminal Preview 1.16.2641.0 and am still not seeing "Open in Terminal Preview" anywhere except when RMBing on a folder. Does not appear when clicking in the background of a folder, or on the desktop, which both should theoretically be fixed according to the commit
From Win10 20H2


UnicodeDecodeError on Python2

I'm trying to install wheel on Python 2 and getting an UnicodeDecodeError  caused by a non-ASCII filename in the test tree of the package, see https://github.com/pypa/wheel/issues/331#issuecomment-586741410. When debugging this I found that this is likely a bug in this repo.
Short: This appeared after some change while it worked with the same file before.
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    setup(maintainer=u'Alex Grnholm')
  File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 145, in setup
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/install.py", line 67, in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/install.py", line 109, in do_egg_install
  File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 204, in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 321, in copy_metadata_to
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 40: ordinal not in range(128)

I found the change which led to the issue: https://github.com/pypa/wheel/commit/e29a5bdd0817f4068aac83edf110caf7c421f574
It adds package_dir={'': 'src'} which is picked up by setuptools as type unicode which leads to an egg-info folder name of type unicode while it was str before. See https://github.com/pypa/setuptools/blob/v44.0.0/setuptools/command/egg_info.py#L218 and https://github.com/pypa/setuptools/blob/v44.0.0/setuptools/command/egg_info.py#L223
This leads to a mix of str and unicode paths at https://github.com/pypa/setuptools/blob/v44.0.0/setuptools/command/bdist_egg.py#L321 (path is str, prefix is unicode) and then this issue when trying to convert one to the other which it didn't before (as both were str)
Will there be any bugfix release for Python2? Is any workaround possible from side of the wheel package?
To reproduce:
I'm using Python 2.7.5 to install setuptools 44.0.0, then wheel 0.34.2. Both done by downloading the tarball from PyPi and running:

python setup.py build
python setup.py install --prefix=<somedir>

A possible solution I found is to add a self.egg_info = self.egg_info.encode('utf-8') after https://github.com/pypa/setuptools/blob/v44.0.0/setuptools/command/egg_info.py#L223 possibly guarded by if six.PY2:

Will there be any bugfix release for Python2?

Setuptools will support bugfix releases against the maint/44.x branch as long as they're warranted (worth the trouble of cutting a release). The use-case you describe seems like a reasonable one to support. I'm surprised this issue is only just now being reported, given that src directories have been supported in setup.cfg for some time.

Interestingly, the issue doesn't happen when using pip to build/install wheel:
draft $ python2.7 -m pip install --no-binary :all: -t temp wheel
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting wheel
  Using cached wheel-0.34.2.tar.gz (58 kB)
Skipping wheel build for wheel, due to binaries being disabled for it.
Installing collected packages: wheel
    Running setup.py install for wheel ... done
Successfully installed wheel-0.34.2

I'm not sure setup.py install should be supported.

The use case runs this on the extracted source tree. This is used because sometimes patches are required.
I don't understand why setup.py install should not be supported. Isn't that how to install packages from source?

I don't understand why setup.py install should not be supported. Isn't that how to install packages from source?

No, the preferred way to install packages is with pip; in the case of installing it from the repo root, you'd use pip install ..

I'm not sure setup.py install should be supported.

I agree with that. I think generally our policy has been "if pip install works, we should advise people to use that." Unless this fix is particularly easy, I'm not sure it's worth the effort of backporting it to 44.x considering it only affects people using a combination of two deprecated technologies: Python 2.7 and setup.py install (though admittedly the Python 2 deprecation has been much louder than the setup.py install deprecation).

In our case we use it to install pip: first setuptools, then wheel, then pip, see https://github.com/easybuilders/easybuild-easyconfigs/pull/9859 (look at changes, you'll get the gist)
so at this point we need the setup.py.
Anyway I'm surprised pip works. Doesn't it also run setup.py at some point?

I'd recommend not installing pip that way. The pip instructions recommend installing using get-pip.py, which bypasses the setuptools build step. By using setup.py install, you're invoking easy_install, so you're getting older and deprecated behaviors.

Anyway I'm surprised pip works. Doesn't it also run setup.py at some point?

It does, but it passes other parameters and probably doesn't pass --prefix. And if wheel were to declare its build with PEP 517, setup.py would not be invoked and instead the setuptools would be called directly.

I realize this approach is an inversion from the approach that may have been recommended a decade ago, but things have changed. Get pip first (with the bootstrap script if necessary), then install build tools (if needed; we're working on making setuptools not an implicit requirement for any package).

In the case of installing it from the repo root, you'd use pip install ..

Unfortunately this doesn't work right away. To complete the information:
Goal: Install pip, setuptools and wheel with specified versions into a given prefix using the system python. It should then be usable as a module (basically: set PYTHONPATH and PATH so binaries and packages are found)
With the "old" approach running `setup.py this works (until the update to wheel showing a bug in setuptools).
Now there may or may not be a system installed pip. So following approaches:

python get-pip.py --no-setuptools --no-wheel --prefix /tmp/pip-prefix
--> Fails because it tries to uninstall system pip which it cannot due to permissions
pip install --upgrade --prefix /tmp/pip-prefix pip
--> Same as above
Download wheel tarball from PyPi and run pip install --prefix=/tmp/pip-prefix .
--> Installs "UNKNOWN" (creating pip-egg-info/UNKNOWN.egg-info)

However it seems using -I/--ignore-installed does indeed work. Is this "the valid solution"(TM)?
BTW: Seems that using pip to install the package it doesn't touch the test files so it doesn't encounter the invalid path. And using printf debugging it doesn even run into the function at https://github.com/pypa/setuptools/blob/v44.0.0/setuptools/command/bdist_egg.py#L321
So question: Is there a (valid) use of pip install which is supposed to run into that code?

However it seems using -I/--ignore-installed does indeed work. Is this "the valid solution"(TM)?

Perhaps. If it works, then I'd say use it. Regardless, we've now established that setup.py install should not be used and we'll rely on pip to provide improved bootstrapping as needed. Also, by now, support for Python 2 is pretty much dropped. Thanks for your understanding.


net: document that DialIP can only be used with WriteMsgIP of Connection-less protocols

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.9.2 linux/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ilyaigpetrov/go"
GORACE=""
GOROOT="/usr/lib/go-1.9"
GOTOOLDIR="/usr/lib/go-1.9/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build418566627=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?
WriteMsgIP can't be used after DialIP to TCP
package main

import (
  "net"
  "fmt"
)

func main() {

  loIP := net.IPv4(127,0,0,1)

  // Accept TCP connections on localohst
  listener, err := net.ListenTCP("tcp", &net.TCPAddr{ IP: loIP , Port: 0})
  if err != nil {
    panic(err)
  }
  go func(){
    for {
      _, err := listener.Accept()
      if err != nil {
        fmt.Println(err)
      }
    }
  }()

  // Dial localhost
  ipconn, err := net.DialIP("ip:tcp", nil, &net.IPAddr{IP: loIP})
  if err != nil {
    panic(err)
  }
  _, _, err = ipconn.WriteMsgIP([]byte{11,22,33}, []byte{}, &net.IPAddr{ IP: loIP })
  if err != nil {
    panic(err)
  }
  fmt.Println("Heppy exit!")

}

What did you expect to see?
It doesn't work. Error appears: write ip <IP_ADDRESS>-><IP_ADDRESS>: use of WriteTo with pre-connected connection.
It seems DialIP can't be used with WriteMsgIP and I want this to be documented.
What did you see instead?
No documentation.

/cc @mikioh


Improve code gen to emit debug symbols

Improve code gen to emit line/column information in debug symbols. Implementation in the VM is done already.
Depends on #3 and #4.

Zig based compiler emits debug symbols with correct line offsets


feat(devtools): add devtools web component

Pull Request Test Coverage Report for Build<PHONE_NUMBER>0
Details

146 of 234 (62.39%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-4.2%) to 95.8%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/devtools/src/component.ts
88
104
84.62%

packages/devtools/e2e/index.ts
0
72
0.0%

Totals

Change from base Build<PHONE_NUMBER>7:
-4.2%

Covered Lines:
1608

Relevant Lines:
1696

💛  - Coveralls


Input success is incorrect

Input success should be the green success color, not blue. Semi-related to #19

I'll go ahead and submit a pull request for this one!

#52

Sorry, I made a mistake 😱
I reopened it. 🙇

@BcRikko Should be good!

I merged #56 🙏


Cop out on fixing a spurious test failure

This commit cops out trying to fix rename_with_link_search_path by simply
adding a loop on Windows to retry the operation that looks to need retrying.

r? @matklad
(rust_highfive has picked a reviewer for you, use r? to override)

@bors r+

:pushpin: Commit 1759b30 has been approved by matklad

:hourglass: Testing commit 1759b30cf713daf66fda5d27499875af4d110cf5 with merge 0f9df98459288a6b296e052c38997153e742f673...

Awesome, thanks @ehuss! Hopefully we don't have to put this logic in too many places...

:sunny: Test successful - status-appveyor, status-travis
Approved by: matklad
Pushing 0f9df98459288a6b296e052c38997153e742f673 to master...


[Feature Request] Trigger validation programatically

Is your feature request related to a problem? Please describe.
If a user doesn't enter a value, the validation doesn't happen until submission. However, if I am in a wizard (a single form has multiple steps), I want to prevent someone from continuing to the next step. If someone clicks the next button, the fields are not checked so the user cannot be prevented.
Describe the solution you'd like
I'd like a function returned by useForm to trigger a validation of the fields. Bonus points to also accept string | string[] fields to only validate given fields.
Describe alternatives you've considered
Currently I have to also use getValues but this doesn't show the error message then:
validate: (): boolean => !errors.title && Boolean(getValues().title),

I'd like to do:
validate: (): boolean => validate('title')

And the errors object then have an error (if the field is invalid) so that my form can show the error also. From what I've seen in the code, the validation is only internal and no way for me to get to it easily and the different modes don't handle if the field was never focused.

I like this validate() function, will be working on that.
in terms of validated specific fields, there is another users reported the same concern as well. I think it makes sense to specify fields to validate at useForm level.

I have quickly build a beta version, maybe you can help to test, I will do my own after work.
useForm({ validationFields: string[] })

validate() => boolean

<EMAIL_ADDRESS>

validateField(): validate individual fields
https://codesandbox.io/s/react-hook-form-validatefield-w1g0m
validationFields: restrict fields with validation
https://codesandbox.io/s/react-hook-form-validationfields-1xb91
still in the beta will publish soon

Thank you @bluebill1049. The trigger function is particularly what I want.

And fyi, I have used 3.9.0-beta.7 for testing and the trigger function is what is working for me.

cool, i will publish soon

published: https://github.com/bluebill1049/react-hook-form/releases/tag/v3.9.0 🤗

This great - just wondering why you didn’t stick with validate()?

because  register already have validate method, i dont want it to be confused with both.
regsiter({ validate: (data) => data === 'something' });
that's why trigger popped in my head to trigger validation.

Thanks for your work on this so fast and your work on this module @bluebill1049

no worries @mitchellsimoens, please consider starring the repo to support and share around the community :) 🙏

hey @mitchellsimoens,
Quick question here, you happy for trigger to be able to validate onlyafter submit button pressed?
https://github.com/bluebill1049/react-hook-form/pull/60
cheers
bill

@bluebill1049 as long as I can programmatically trigger a validation, I'm happy. Even if it's not technically a public api, so long as I have access to mess around a bit.

FYI: I have changed the name of the method from 'trigger' to 'trigger validation'. but won't affect your current version: 3.10.2
https://react-hook-form.com/api


[REVIEW] Avoid empty gdf_table creation in joins

Fixes pygdf#281
In the process of creating the complete dataframe while joining three gdf_tables were instantiated :

Left columns that are not joined on.
Common columns that are joined on.
Right columns that are not joined on.

This fix instantiates 1 or 3 only if the number of columns in the table are non zero and calls gather.

@kaatish This PR reminded me that we should also add a check if the number of columns to join on is 0, we should just return immediately with GDF_SUCCESS. Could you add that to this PR as well.

@kaatish This PR reminded me that we should also add a check if the number of columns to join on is 0, we should just return immediately with GDF_SUCCESS. Could you add that to this PR as well.

Added.

rerun tests


Jinja templating seems to be crashing PyStack with an IndexError

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
<irrelevant stack trace output truncated>
    (Python) File "/opt/venvs/debtsy/lib/python3.9/site-packages/flask/templating.py", line 120, in _render
        rv = template.render(context)
    (Python) File "/opt/venvs/debtsy/lib/python3.9/site-packages/ddtrace/contrib/jinja2/patch.py", line 70, in _wrap_render
        return wrapped(*args, **kwargs)
    (Python) File "/opt/venvs/debtsy/lib/python3.9/site-packages/jinja2/environment.py", line 1043, in render
        return concat(self.root_render_func(self.new_context(vars)))
    (Python) File "/opt/venvs/debtsy/lib/python3.9/site-packages/flask_admin/templates/bootstrap3/admin/model/edit.html", line 24, in root
        {%- if admin_view.can_view_details -%}
    (Python) File "/opt/venvs/debtsy/lib/python3.9/site-packages/flask_admin/templates/bootstrap3/admin/master.html", line 16, in root
Traceback (most recent call last):
  File "/root/venv/bin/pystack", line 8, in <module>
    sys.exit(main())
  File "/root/venv/lib/python3.8/site-packages/pystack/__main__.py", line 254, in main
    args.func(parser, args)
  File "/root/venv/lib/python3.8/site-packages/pystack/__main__.py", line 381, in process_core
    print_thread(thread, native)
  File "/root/venv/lib/python3.8/site-packages/pystack/traceback_formatter.py", line 15, in print_thread
    for line in format_thread(thread, native):
  File "/root/venv/lib/python3.8/site-packages/pystack/traceback_formatter.py", line 81, in format_thread
    yield from format_frame(current_frame)
  File "/root/venv/lib/python3.8/site-packages/pystack/traceback_formatter.py", line 29, in format_frame
    source = lines[code.location.lineno - 1]
IndexError: list index out of range

Expected Behavior
PyStack does not crash when processing this process.
Steps To Reproduce
I can try and nail down a more constrained STR, but I'm not really sure what to actually try since that it's hard to reduce our entire app to a minimal example... given the traceback (see below), I figured I would file a bug and see if anyone had any ideas on what to try based on their knowledge of the way PyStack works.
PyStack was freshly installed from Pip FWIW, into a fresh virtual environment. This crash happened on both a coredump file and on a live process (processing a request for the same URL). We're running Ubuntu 20.04.6.
Pystack Version
1.3.0
Python Version
3.8
Linux distribution
Ubuntu
Anything else?
No response

Hi @strugee and thanks for the report! We can try to investigate but without a reproducer is very hard for us to do anything here. Can you try to get a small jinja-based example to see if it crashes?
if not, can you modify /root/venv/lib/python3.8/site-packages/pystack/traceback_formatter.py to print lines and code.location? Like so:
print(lines, code.location) source = lines[code.location.lineno - 1]


how to create a model class for Salesforce custom object

I are trying to learn the Django-salesforce python library and  I am new to python Django and salesforce. I follow the following GitHub library https://github.com/django-salesforce/django-salesforce and I follow the following step https://pypi.org/project/django-salesforce/ in python. and I got successfully run my project, But now I am confused because I can't access the custom field of salesforce object can you tell me how can I access the custom field of standard objects like account, contact, etc. I try it but I got following error
I have a query regarding the below point.
Here is my code :
class Contact(SalesforceModel):
account = models.ForeignKey(Account, on_delete=models.DO_NOTHING,
                            blank=True, null=True)  # db_column: 'AccountId'
name = models.CharField(max_length=121, sf_read_only=models.READ_ONLY,
                        verbose_name='Full Name')
Loan_Amount__c = models.IntegerField(max_length=121, blank=True, null=True)

def __str__(self):
    return self.name

ERRORS:
webapp.Contact.Loan_Amount__c: (fields.E002) Field names must not contain "__".
can you tell me how can I access the custom field of salesforce standard object or can give a sample code for this ?.
and I have 1 more question about how can we create a model class for custom Salesforce objects. I have created a class for the custom object but it shows me an error.
class NewClient__c(SalesforceModel):
ClientName = models.CharField(max_length=255)
Name = models.CharField(max_length=121, sf_read_only=models.READ_ONLY,
                        verbose_name='Full Name')
email = models.EmailField(blank=True, null=True)

def __str__(self):
    return self.Name

whenever I put __c in the class name it shows me the following error. Without __c I can communicate with salesforce objects its mandatory for salesforce ?.
ERRORS:
webapp.NewClient__c: (models.E024) The model name 'NewClient__c' cannot contain double underscores as it collides with the query lookup syntax.
can help me solve this error or do you have a and sample code to retrieve data from a custom object in Django.

Field names and class names can not contain two consecutive underscores in Django, because '__' has a special meaning in queries, very similar to a dot expressions.
See https://docs.djangoproject.com/en/3.0/topics/db/models/#field-name-restrictions
The fix is easy
class Contact(SalesforceModel):
    ...
    loan_amount = models.IntegerField(max_length=121, db_column='Loan_Amount__c', blank=True, null=True)

class NewClient(SalesforceModel):
    ....
    Meta:
        db_table = 'NewClient__c'


Use Into<Option<&str>>

Part of https://github.com/gtk-rs/gir/pull/314, #438
WIP

Closing in favor of #447.


Active MultipleEmitters twitter/compose-rules

Issue

close #515

Overview (Required)

active MultipleEmitters rule of compose-rules

No lint error.

Links

Ruleset

Great!


Fix for issue #207

Created a new object graph visitor for yaml dotnet to ignore default and also empty collections. Unifiy creation of serializer / deserializer well configured.

LGTM.


Move dependencies to their own shards to share code with Gtk3 bindings

Gtk4 and 3 share a lot of dependencies in common, to avoid repeat the work already done in this shard on gtk3 shard, common dependencies must be moved to their own shards.

[ ] gio-2.0
[x] harf_buzz-0.0
[ ] pango-1.0

Binding that don't need (yet) a bindings.yml file don't need a shard.

harfbuzz shard created at https://github.com/hugopl/harfbuzz.cr.
BTW I noticed that the generated bindings for HarfBuzz library are 100% unusable, for now they only exists to make GTK shards compile.


scenario outlines can now use examples table in step definition tables

Thanks for the PR! I tested both SpecFlow and Cucumber.js and saw that using example data within step argument tables is supported just as you describe. I wasn't aware that this was even possible in Cucumber, but it makes sense.
The logic in your PR looks great. However, there are a few related concerns:

This project doesn't currently have TSLint and Travis CI setup, so I opened #6. There are some style differences in this PR from the rest of the codebase that TSLint would have identified if I had bothered to use it. Specifically, the rest of the codebase uses const and let instead of var, uses arrow functions instead of the function keyword, and has no newlines before open curly braces.

(optional) I discovered that that using example table values is supported in docstrings as well. jest-cucumber currently supports docstrings, so it would be good to update that as well for consistency. If you don't have the time / interest to address this as part of this PR, no worries--I can just open up a new issue.

(optional) The examples in this project serve as both documentation for each feature in both TS and ES, and also as integration tests. Since we're adding additional behaviors to scenario outlines (using example table data in docstring and table arguments), it would be good to have additional examples as well. This could also be addressed in #6, since there are other features that aren't currently documented in the examples--case in point, docstrings. I should probably document this in the contribution guidelines, which this project also currently lacks.

Just pushed out those changes, I worked from the javascript and then transferred to the typescript but wasn't very thorough! anyway I think I've address the comments in 1)
If I get some time I'll look at 2) & 3) also

Thanks--looks great! Merged and created #7 for 2) above. Your changes are now published in v1.0.6 on NPM.


Add field in models for access point of archived URL

@vphill requested a field in the Project? model (or a set of fields) that would allow us to put the base URL for an OpenWayback instance that provides access to the seeds crawled in the Nomination Tool.
If that seed is present then we could have a link directly into the archive created for it.
Might make the nomination tool an access system as well.
(issue moved over from Trac)

Would this link be something we'd want to be displayed in each Project's About page, if the link was present for that project?

yes

I think the plan might also be to use that base URL to construct links for urls/seeds to where they exist for playback in external OpenWayback instances. This could possibly be displayed on the /url views. Is that part of the plan @vphill ?

Yes that is also a desire.  So the nomination tool could be used as an access tool once a based openwayback access point is established for a project. @ldko can give info on the best way to construct the link into openwayback.  I think the calendar view for a given URL is the way to go.

For example, if the nominated url in the project is:
http://www.unt.edu
and the base url for the OpenWayback instance is:
http://webarchive.library.unt.edu/unt/fall2015/
the url we would generate to link to the archived url in OpenWayback would be:
http://webarchive.library.unt.edu/unt/fall2015/*/http://www.unt.edu
or if the nominated url is:
http://www.artlies.org
and the base url for the OpenWayback instance is:
http://webarchive.library.unt.edu/artlies/
the url we would generate to link to the archived url in OpenWayback would be:
http://webarchive.library.unt.edu/artlies/*/http://www.artlies.org

gotcha. should be easy enough to do


Need Help

Trying to do the same thing as this repo, but it won't let me. Got denied by github-actions, maybe you could help me with it. Thank you. here's the log
remote: Permission to codingjeff01/auto-update.git denied to github-actions[bot].
fatal: unable to access 'https://github.com/codingjeff01/auto-update/': The requested URL returned error: 403
Error: Invalid status code: 128
    at ChildProcess.<anonymous> (/home/runner/work/_actions/stefanzweifel/git-auto-commit-action/v4/index.js:17:19)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5) {
  code: 128
}
Error: Invalid status code: 128
    at ChildProcess.<anonymous> (/home/runner/work/_actions/stefanzweifel/git-auto-commit-action/v4/index.js:17:19)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

Regarding your issue, make sure you have enabled the Workflow permissions.
Here's how to do it:

Head to settings
Go to Actions > General
Scroll down and see Workflow permissions
- Ensure that Read and write permissions is selected.
- Also, select Allow GitHub Actions to create and approve pull requests

Let me know if this works.

Thank you so much, I have other questions. How do you add the api secrets?

Since this repo uses GitHub Actions:

Go to Settings.
Navigate to Secrets and Variables > Actions
Press New repository secret
Input the Name WEATHER_API_TOKEN and put your api key on secret

If you have any further queries or need additional assistance, feel free to ask here.

Thank you so much! I've resolved all the issues I was facing. Thank you for your assistance. I will close these issues now, as I haven't encountered any errors or problems.


Sizes between inversions do not correspond across the genome

Hello! I obtained figures comparing three genomes using minimap2, SyRI, and plotsr. However, there seem to be issues where the sizes between inversions do not correspond and the representation of synteny is not detailed enough and appears too clustered. Is there a specific part of the command that I should modify to address these problems?

Hello! I came across the same problem with the sizes between inversions not corresponding. I was trying to search through the tool on the usage of --invgaplen, but I cannot find it. What do you specify as an argument? And how can I filter out inversions with very different sizes? Thank you for your help!

I was trying to search through the tool on the usage of --invgaplen, but I cannot find it. What do you specify as an argument?

--invgaplen is a parameter for syri. So you would need to rerun syri with this parameter if you do want inversions with large indels.

And how can I filter out inversions with very different sizes?

Rerun syri with the --invgaplen parameter. Or you can filter the input files for plotsr (for example: syri.out file) to remove such inversions.

Thank you so much for your response! Yes, I wanted to rerun syri. But my question is what do you specify as the argument for the parameter "--invgaplen"? Is it the size you do not want the inversions to exceed?

Output from syri  -h:
 --invgaplen INVGL     Maximum allowed gap-length between two alignments of a multi-alignment inversion. It affects the selection
                        of large inversions that can have different length in the reference and query genomes. (default:
<PHONE_NUMBER>)

Thank you so much! I am sorry, I have not noticed it there.


Checking if an application is running under Burrito

I want to be able to fetch arguments from my Application, regardless of if the application is being run from a burrito-based release or with mix. Ideally, I'd be able to do something like this
def get_args() do
  if Burrito.is_standalone?() do
    Burrito.Util.Args.get_argments()
  else
    System.argv()
  end
end

Is there a way to do this that I'm not seeing? If not, consider this issue a feature request :)

I have the same issue as I support building with escript (which I use locally) and Burrito (which I intend for users).
I'm not aware of a built in way to do that so I currently solve this by passing an environment variable BUILD_MODE=[escript|burrito] to configure the project definition and how args are looked up.

I'm thinking of passing an extra env variable into the erlang launcher environment, something like __BURRITO=1 -- would that suffice?

I just release Burrito v1.0.3 to Hex! Now when a burrito app is launched, the environment variable __BURRITO will have the value 1. You can use this at runtime to check if your app is running inside burrito!


Add azurerm_federated_identity_credential resource

1664
PR Checklist

[x] I have added example(s) inside the [./examples/] folder
[x] I have added the example(s) to the integration test list for normal (~30 minutes) or long runner >30 minutes
[x] I have checked the coding conventions as per the wiki
[x] I have checked to ensure there aren't other open Pull Requests for the same update/change?

Description

Does this introduce a breaking change

[x] YES
[ ] NO

Changes in the code support working with azurerm_federated_identity_credential resource.
This is required for replacing deprecated AAD Pod Identity with Azure AD workload identity with Azure Kubernetes Service
Testing

Thanks for the PR @mdanylyuk, can you please add an example which can be fully tested in standalone tests (add an AKS cluster, or a scenario that can be self-sufficient).
When testing your code, getting the following:
│ Error: Invalid index
│
│   on ../managed_identities.tf line 36, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   36:   issuer              = each.value.issuer != null ? each.value.issuer : local.combined_objects_aks_clusters[try(each.value.aks_cluster.lz_key, local.client_config.landingzone_key)][each.value.aks_cluster.key].oidc_issuer_url
│     ├────────────────
│     │ each.value.aks_cluster.lz_key is "aks"
│     │ local.client_config.landingzone_key is "examples"
│     │ local.combined_objects_aks_clusters is object with 1 attribute "examples"
│
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│
│   on ../managed_identities.tf line 36, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   36:   issuer              = each.value.issuer != null ? each.value.issuer : local.combined_objects_aks_clusters[try(each.value.aks_cluster.lz_key, local.client_config.landingzone_key)][each.value.aks_cluster.key].oidc_issuer_url
│     ├────────────────
│     │ each.value.aks_cluster.lz_key is "platform"
│     │ local.client_config.landingzone_key is "examples"
│     │ local.combined_objects_aks_clusters is object with 1 attribute "examples"
│
│ The given key does not identify an element in this collection value.
╵
Terraform plan return code: 1
Error 1 on or near line 386: Error running terraform plan; exiting with status 1

Thanks for the PR @mdanylyuk, can you please add an example which can be fully tested in standalone tests (add an AKS cluster, or a scenario that can be self-sufficient).
When testing your code, getting the following:
│ Error: Invalid index
│
│   on ../managed_identities.tf line 36, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   36:   issuer              = each.value.issuer != null ? each.value.issuer : local.combined_objects_aks_clusters[try(each.value.aks_cluster.lz_key, local.client_config.landingzone_key)][each.value.aks_cluster.key].oidc_issuer_url
│     ├────────────────
│     │ each.value.aks_cluster.lz_key is "aks"
│     │ local.client_config.landingzone_key is "examples"
│     │ local.combined_objects_aks_clusters is object with 1 attribute "examples"
│
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│
│   on ../managed_identities.tf line 36, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   36:   issuer              = each.value.issuer != null ? each.value.issuer : local.combined_objects_aks_clusters[try(each.value.aks_cluster.lz_key, local.client_config.landingzone_key)][each.value.aks_cluster.key].oidc_issuer_url
│     ├────────────────
│     │ each.value.aks_cluster.lz_key is "platform"
│     │ local.client_config.landingzone_key is "examples"
│     │ local.combined_objects_aks_clusters is object with 1 attribute "examples"
│
│ The given key does not identify an element in this collection value.
╵
Terraform plan return code: 1
Error 1 on or near line 386: Error running terraform plan; exiting with status 1

Thanks for the comment,
I added new resources, VNet and AKS, in the example - please check one more time.

hi @mdanylyuk, thanks for the update, im now getting the following error:
╷
│ Error: creating Federated Identity Credential (Subscription: ""
│ Resource Group Name: "bfqy-rg-security-rg1"
│ User Assigned Identity Name: "bfqy-msi-poc-mi"
│ Federated Identity Credential Name: "bfqy-msi-dev"): managedidentities.ManagedIdentitiesClient#FederatedIdentityCredentialsCreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="The request format was unexpected. Federated Identity Credential must specify 'issuer', 'subject' and 'audience'."
│
│   with module.example.azurerm_federated_identity_credential.fidc_aks["dev_poc_msi_region1"],
│   on ../managed_identities.tf line 29, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   29: resource "azurerm_federated_identity_credential" "fidc_aks" {
│
│ creating Federated Identity Credential (Subscription: ""
│ Resource Group Name: "bfqy-rg-security-rg1"
│ User Assigned Identity Name: "bfqy-msi-poc-mi"
│ Federated Identity Credential Name: "bfqy-msi-dev"): managedidentities.ManagedIdentitiesClient#FederatedIdentityCredentialsCreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned
│ an error. Status=400 Code="BadRequest" Message="The request format was unexpected. Federated Identity Credential must specify 'issuer', 'subject' and 'audience'."
╵
╷
│ Error: creating Federated Identity Credential (Subscription: ""
│ Resource Group Name: "bfqy-rg-security-rg1"
│ User Assigned Identity Name: "bfqy-msi-poc-mi"
│ Federated Identity Credential Name: "bfqy-msi-qa"): managedidentities.ManagedIdentitiesClient#FederatedIdentityCredentialsCreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="The request format was unexpected. Federated Identity Credential must specify 'issuer', 'subject' and 'audience'."
│
│   with module.example.azurerm_federated_identity_credential.fidc_aks["qa_poc_msi_region1"],
│   on ../managed_identities.tf line 29, in resource "azurerm_federated_identity_credential" "fidc_aks":
│   29: resource "azurerm_federated_identity_credential" "fidc_aks" {
│
│ creating Federated Identity Credential (Subscription: ""
│ Resource Group Name: "bfqy-rg-security-rg1"
│ User Assigned Identity Name: "bfqy-msi-poc-mi"
│ Federated Identity Credential Name: "bfqy-msi-qa"): managedidentities.ManagedIdentitiesClient#FederatedIdentityCredentialsCreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned
│ an error. Status=400 Code="BadRequest" Message="The request format was unexpected. Federated Identity Credential must specify 'issuer', 'subject' and 'audience'."
╵
Terraform apply return code: 1
Error 1 on or near line 110: Error running terraform apply; exiting with status 1

bfqy-msi-poc-mi

Could you please attach tfplan to see the whole changes?

mdanylyuk-md/federated-identity-credential

@mdanylyuk, the plan works fine, its an apply time error. It is easy to repro, I'm just running the example here :)
Also, you are missing the following outputs in aks
output "oidc_issuer_url" {
  value = azurerm_kubernetes_cluster.aks.oidc_issuer_url
}

output "oidc_issuer_url"

@arnaudlh, many thanks! It's really my mistake (sorry but I worked in the multiple repos with it and missed adding)
Output is added now


chore: remove lockfile removal in script, prevent unnecessary installation #trivial

This PR resolves []
Description

Noticed gems get installed every time I run the release scripts when they should only need to be installed once.
PR Checklist

[ ] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Don't install gems unnecessary for fastlane scripts - Brian

Need help with something? Have a look at our docs, or get in touch with us.

Android caching is a bit of a mystery to me unfortunately but it does seem like it is not working well some of the mobile-practice folks may have a better idea


stakeValidator.CheckKernel should use return codes, not exceptions (stops API from responding)

The current implementation of CheckKernel relies on exceptions for normal processing, and not return codes. It is throwing a huge amount of exceptions when processing. It does this so much, that the API ends up becoming unresponsive for the wallet. The API starts throwing errors short time after enabling staking.
Changing the implementation of CheckKernel to return status codes (int), fixes the issues. I have tested for a few minutes, and the API responds with no issues in less than 100 ms for most requests.
With exceptions being thrown in this method, then requests starts taking 3+ seconds and fails.
Here is network log, showing what happens during the kernel execution:

And sometimes it is total stop:

Individual loop for each of the 30 threads that is spawned for POS "mining", is reduced from 6692346-185683984 ticks to only 20-5000 ticks. Huge performance increase by not relying on exceptions for normal processing (it checks for a specific exception and then continues to process the loop).
My suggestion is to do something similar to:
var result = this.stakeValidator.CheckKernel(contextInformation, chainTip, block.Header.Bits, txTime, prevoutStake);
And have CheckKernel return status code (integer) for the various steps in the validation.
Discovered this when I was attempting to debug why staking (not on Stratis network) wouldn't work properly, and I kept getting errors from the API whenever staking was enabled.
The CheckKernel is processed to the very end, this is where it "fails":
// Now check if proof-of-stake hash meets target protocol.
var hashProofOfStakeTarget = new BigInteger(1, context.HashProofOfStake.ToBytes(false));
if (hashProofOfStakeTarget.CompareTo(weightedTarget) > 0)
{
    this.logger.LogTrace("(-)[TARGET_MISSED]");
    ConsensusErrors.StakeHashInvalidTarget.Throw();

The wallet has more than 700 transactions which is part of the staking.

Could also be a simple boolean return value, as it is only a single exception type that performs continue, the others are exceptions and should probably just be?

Performed some performance logging, the following snippet is what I timed a difference between boolean return value, and exception:
var workersResult = new CoinstakeWorkerResult();
for (int workerIndex = 0; workerIndex < workerCount; workerIndex++)
{
    var cwc = new CoinstakeWorkerContext
    {
        Index = workerIndex,
        Logger = this.loggerFactory.CreateLogger(this.GetType().FullName, $"[Worker #{workerIndex}] "),
        utxoStakeDescriptions = new List<UtxoStakeDescription>(),
        CoinstakeContext = coinstakeContext,
        Result = workersResult
    };

    int stakingUtxoCount = Math.Min(stakingUtxoDescriptions.Count - coinIndex, UtxoStakeDescriptionsPerCoinstakeWorker);
    cwc.utxoStakeDescriptions.AddRange(stakingUtxoDescriptions.GetRange(coinIndex, stakingUtxoCount));
    coinIndex += stakingUtxoCount;
    workerContexts[workerIndex] = cwc;

    workers[workerIndex] = Task.Run(() => this.CoinstakeWorker(cwc, chainTip, block, minimalAllowedTime, searchInterval));
}

await Task.WhenAll(workers).ConfigureAwait(false);

Handling exception:
14,694 ms
45,503 ms
36,132 ms
28,956 ms
Handling return value:
8,782 ms
429 ms
170 ms
3,032 ms
154 ms
145 ms
This is on my Core i-7 8700K, 16GB RAM, M.2 SSD PC, so numbers should be a lot higher on smaller devices, if for example someone wants to stake on Raspberry Pi or lowlevel/cheap cloud VM.

Interesting idea, normally on the consensus path stake is valid so no perf issue but in stake path perhaps

I don't know for sure, but from debugging I don't think that the minting process actually does any more attempts even if it runs faster. Nonce is not used, instead it's a combination of transaction inputs and time values that are random inputs for block generation, or what?
When the difficulty is high then (due to POW making it high), it might not be able to easily find POS blocks by minting, and searchInterval is calculated perhaps too low to perform enough attempts?
Without changing the return value of CheckKernel, to use boolean return value, the CPU will spike for a short time each time it attempts to find a new block.

Ok that's a good reason to do it already.
Can you share your PR lets give it a look.


Can't link to github

Extension version: 0.8.0
VSCode Version: 1.35.1
Electron: 4.1.4
OS: Linux x64 5.1.9-arch1-1-ARCH

Steps to Reproduce:

Upon installing the plugin from the marketplace, I opened the prompt asking me to sign to Github. I signed, and asked me to authorize it. I clicked on Authorize, and i got this - https://gyazo.com/30ccad74c0979101ea4525dee79b7fbf , In the URL there was code-oss:// which I guess isn't found as an application, no idea why. After this, it was idle on signing to Github.com in the IDE. I checked my OAuths in Github, and it wasn't there either.

@TcpAckFrequency Are you using the arch linux community package, or the normal download of VS Code?

Edit: The OAuth is there. Im using the Arch linux community package.

Thanks for confirming. I think I understand what's going on here, I'm not sure what we can do to fix it from this side.
For each version of vscode, it ships with a file with some product configurations, including the uri scheme that should be used for protocol handlers. For the normal VS Code download, this is vscode, for the Arch linux package it looks like this is code-oss. The extension uses whatever this value is to construct the callback URL.
After sign in fails, can you try running the command GitHub Pull Requests: Manually Provide Authentication Response and pasting the data that was on the auth page?

Okay fixed it by adding a new token and manually pasting it. Thanks.

@RMacfarlane could you elaborate on the solution you would have suggested to @TcpAckFrequency, I also on arch am having issues getting set up. logging in with ctrl-p GitHub Pull Requests: Sign in to GitHub  does not login, the signin button in the sidebar does not open a browser to authenticate through, and manually pasting in credentials are deleted on refresh.

@alienfetuseater There are some steps for this here: https://github.com/microsoft/vscode-pull-request-github/wiki#personal-access-token-authentication
By "manually pasting in credentials are deleted on refresh", do you mean that you that you are signed out again when VSCode is reloaded?

On Thu, Oct 10, 2019 at 12:50 PM Rachel Macfarlane<EMAIL_ADDRESS>wrote:

@alienfetuseater https://github.com/alienfetuseater There are some
steps for this here:
https://github.com/microsoft/vscode-pull-request-github/wiki#personal-access-token-authentication
By "manually pasting in credentials are deleted on refresh", do you mean
that you that you are signed out again when VSCode is reloaded?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-pull-request-github/issues/1244?email_source=notifications&email_token=AJJ3UH2CBHHRAO6DCKI22Q3QN5MMBA5CNFSM4H2EEPZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA5A2VA#issuecomment-540675412,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJJ3UH64IRLR356J7E7LIWTQN5MMBANCNFSM4H2EEPZQ
.

By deleted I mean the token I paste into settings
“GitHubPullRequest.login”: {[
Server: github.com,
User: ‘myUserName’,
Token: ‘myGeneratedToken’
]},
Is deleted from the settings.json on reload and I never got logged in. It
says at the bottom toolbar “GitHub signing in...” or what not

also I tried launching vscode from command line with the option code --enable-proposed-api GitHub.vscode-pull-request-github  .  vscode code starts and the prompt box appears that says
In order to use the Pull Requests functionality, you must sign in to github.com which i click but nothing happens. no browser opens up initiating oauth, only a message on the bottom toolbar, as i said above Signing in to github.com... with the github logo, but nothing happens. When i click on that message on the bottom toolbar, I am prompted to input my token, which i do, and followed by the server (I assume this is to be left as github.com? i tried both github.com and and github.com/myusername/myrepo) but nothing happens the message at the bottom remains the same as before

By deleted I mean i edit settings.json to reflect the following:
   {
     "workbench.iconTheme": "vscode-icons",
        "githubPullRequests.hosts": [
           {
               "host": "https://github.com",
                "token": "a4549d9558cd8549e6fa20530895f518db921010"
            }
        ],
        "editor.fontLigatures": true,
    }

then i reload the window and the above turns into:
{
 "workbench.iconTheme": "vscode-icons",
    "editor.fontLigatures": true,
}

immediately after this deletion from settings.json the pop up in the lower right corner that says In order to use the Pull Requests functionality, you must sign in to github.com appears.


Fix database typeahead in SQL Lab

The typeahead for databases is currently not working in SQL Lab. This happens because the options passed to the component are missing a label attribute, which is used to power the typeahead.
I fixed this by adding a label composed of the database backend and name. This allows users to choose a database by typing its name, eg, "main", or by typing a backend type, eg, "mysql".
I also made the database async endpoint more robust when no databases are found. I came across an error while testing and decided to fix it, since it's somewhat related.

Codecov Report

Merging #6917 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #6917      +/-   ##
==========================================
- Coverage   63.85%   63.85%   -0.01%
==========================================
  Files         421      421
  Lines       20447    20449       +2
  Branches     2220     2220
==========================================
  Hits        13057    13057
- Misses       7258     7260       +2
  Partials      132      132

Impacted Files
Coverage Δ

superset/assets/src/components/TableSelector.jsx
84.55% <0%> (ø)
:arrow_up:

superset/views/core.py
74.91% <0%> (-0.11%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f96b0c...1fd5dcb. Read the comment docs.


Yaml parsing assumes nested values for non-string types

Assume you have  a secret in the following form:
mypassword
---
user: 1234
url: http://my.url.com

The user yaml value cannot be obtained since gopass is complaining about nested values. This is due to the interface assertion in secret.go. Non string types, like bool, float and int are not covered here.
Several fixes are thinkable:

parse with regexp instead of yaml. This would be robust but would prevent extending yaml features in the future.
implement conversion to string in the yaml parser. That would make it a string independent on the method you're using to access the value. It also works with values like
Extend the Value method to support other types. That would work out for most cases but e.g. the conversion to float and back to string could suffer from concatenation: 1.23400 would translate to 1.234 which is probably no transparent to the user. Same goes for 0815
We can also leave it as it is but make clear to the user that those special values must be put in double quotes.

What are your thoughts? I would go for 2. or 3. and can pick up the issue if desired.

The problem is that most people don't understand YAML. And they shouldn't have to. The question I'm asking myself is if it's better have proper YAML support or a proper UX. I'd go for UX here.
IMHO what we should do is to think about dropping the YAML parser. Parsing key-value pairs should be easy and robust even without regexp. I'd only need to think about marshaling and handling of more complex data for a moment, e.g. we use YAML folded style/heresocs in a few secrets.

Just as reference:
Problematic cases are
mypassword
---
user: 1234.131100

If you parse as number, you end up with 1234.1311 as username which is probably wrong for most users.

Thanks for the example. I fully aware of the nature of this problem. I'm just not sure on what the best approach would be.

See also #659


Add market finalization date/time somewhere.

https://github.com/AugurProject/augur-ui/issues/2358
Having finalization time somewhere can help market makers create meta markets based on other markets finalization times.

this is done


Add CAMEO codes for countries as aliases

https://github.com/carrillo/Gdelt/blob/master/resources/staticTables/CAMEO.country.txt

Search variable metadata by CAMEO code
Search variable data by CAMEO code
Return CAMEO code in the get data API
Optional: if the users want to insert data by CAMEO code, then we have to update the Wikifier.

Add CAMEO as country alias
Update API to return CAMEO column if country is present
Update endpoint for Causeex specific behavior

api.add_resource(VariableResource, '/datasets/<string:dataset>/variables/<string:variable>/causeex')

GET /datasets/AMIS/variables/total_supply?country=BEL&include=cameo

I have created two files,

country_cameo_exploded.tsv : cameo codes for countries as alias and cameo code (property: P2010270001)
new_property_cameo_exploded.tsv: label for property P2010270001

Import these files:
python script/import_tsv_postgres.py country_cameo_exploded.tsv
python script/import_tsv_postgres.py new_property_cameo_exploded.tsv

Refresh views
python script/refresh_search_views.py

I have tested that search variable metadata and get data with these cameo codes and it is working fine.
/metadata/variables?country=ETH
/datasets/WDI/variables/access_to_electricity_of_population?country=GAB
ball in your court @zmbq
cameo_code_and_property.zip

Added these files to our list of essential files https://docs.google.com/spreadsheets/d/17V3QMUGdm3TW4vQKcM-Zhm9tNUJVLSX02D1SmF_pF3E/edit#gid=0

@zmbq changes required in queries to fetch CAMEO codes for countries, the property is P2010270001

Many of the cameo codes do not correspond to regions. WSB and ABW, for instance, can't be queried, since their qnode is not associated with any region (they don't have any P31 edge really) - so the API doesn't find them.

Implemented in the itay/cameo branch.

I added the cameo for regions just for completion. I will test the code in the new branch

New update from Dan:
They want to search by and receive the full IRI for CAMEO codes.
/metadata/variables?country=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEOeth
/datasets/WDI/variables/access_to_electricity_of_population?country=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEOgab
In the data returned, they want  the IRIs as well

@zmbq As per the new requirements, I inserted alias and cameo code as IRIs. I am trying to search variable metadata with IRI, not working.
/metadata/variables?country=http%3A%2F%2Fontology.causeex.com%2Fcameo%2FCountryCodeOntology%2FCAMEOeth
also tried
/metadata/variables?country=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEOeth
It is in the database

Updated file with IRIs
cameo_code_and_property.zip

I tested @zmbq changes in the itay/cameo branch. I have to set include=country_cameo for this to work, which is fine.

However, country ids are not included, we want country ids by default.

Fixed in itay/cameo .

As per the new requirements, I inserted alias and cameo code as IRIs. I am trying to search variable metadata with IRI, not working.
/metadata/variables?country=http%3A%2F%2Fontology.causeex.com%2Fcameo%2FCountryCodeOntology%2FCAMEOeth
also tried
/metadata/variables?country=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEOeth
It is in the database

Updated file with IRIs
cameo_code_and_property.zip

@zmbq is this also fixed?

database base containing CAMEO uris https://drive.google.com/file/d/1buKO4UghTGOY_gVnPeW8rW-t4yKnSWiq/view?usp=sharing

This is due to the input sanitation. We do not allow non-alphanumeric characters, so the slashes and sharp signs are removed.
I have fixed the sanitation to also accepts / and #, but this query still fails, since the new cameo codes are not considered aliases. You should add each cameo as two edges - a cameo and an alias edge. Also, note that cameo codes have # in them, so the query above should be for http%3A%2F%2Fontology.causeex.com%2Fcameo%2FCountryCodeOntology%2FCAMEO#eth
I saw you already fixed the colons in tag names.
I have pushed the sanitation fix to the development branch.

You should add each cameo as two edges - a cameo and an alias edge

I did, it was rejected silently by the ON CONFLICT DO NOTHING. Oh well, I recreated the database backup with CAMEO URIs as aliases as well.

Uploaded the database backup to https://drive.google.com/drive/u/1/folders/0AGFpIvVliZecUk9PVA as well

Still can't seem to search with the URIs though, will try in a bit again. In a hurry right now

If it still doesn't work, send me the new database dump and I'll try it.

@zmbq its here https://drive.google.com/drive/u/1/folders/0AGFpIvVliZecUk9PVA

. had to be sanitized as well. Pushed into development.
I'm not crazy about the sanitation relaxations I've added today. I'm not sure whether they introduce a potential SQL injection vulnerability or not. I don't know all the SQL Injection tricks. We still don't allow ' and - characters, which are by far the most common in attacks, but still.

@zmbq I am sorry but this is still not working. Did you test it? what would be magic incantation to make it work?
This is what I am trying
/metadata/variables?country=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEO#blk
and also
/metadata/variables?country=http%3A%2F%2Fontology.causeex.com%2Fcameo%2FCountryCodeOntology%2FCAMEO%23blk
Nothing works, this is after I refresh the search views.

This works if search on alias
/metadata/variables?alias=http://ontology.causeex.com/cameo/CountryCodeOntology/CAMEO#blk

It should work when searching on country. Am I misremembering something?

Causx doesnt want the hash in the URI


Experiment with a special "archive" category

See https://github.com/rust-gamedev/arewegameyet/pull/483
I'll try to make the initial prototype in a week or so (i'm terrible at time estimations though).
Related to https://github.com/rust-gamedev/arewegameyet/issues/95
Related to https://github.com/rust-gamedev/arewegameyet/issues/320

I'm against removing even more stuff from the database, I'd much prefer the archived category to be implemented - https://github.com/rust-gamedev/arewegameyet/issues/485 - though I wasn't able to really work on this yet and won't be in the next couple of weeks. If anyone feels that the need to clean up AWGY is urgent, please feel free to propose PRs that introduce the "archived" category.

So yeah, if anyone wants to work on this now, feel free to propose PRs.

I finally have a bit of free time now, so I had a go at mocking something up for an 'archive' section:

I chose to add it to the existing category pages rather than creating new ones for the sake of simplicity.
Couple of other tweaks I considered:

Rather than having a seperate section, we could just grey them out and sort to the bottom of the list, but not sure whether that would make it clear enough.

I could also grey/red them out even with the layout shown above, just to make them a little more visually distinct.

It could be worth us disabling Crates.io/GitHub data fetching for archived crates, to speed up the build a little.
Not sure if the bed icon makes sense, but I couldn't find a more fitting one 😆

Any thoughts on this?

@17cupsofcoffee (sorry for the late reply!)

I chose to add it to the existing category pages rather than creating new ones for the sake of simplicity.

yeah, it sounds great to me! and I agree that it's worth graying them out anyway too.

It could be worth us disabling Crates.io/GitHub data fetching for archived crates, to speed up the build a little.

most of the libraries don't have a description in our data.toml so if we disable data fetching only a name would left. i'd prefer to keep data fetching for them, at least for now.

Not sure if the bed icon makes sense, but I couldn't find a more fitting one 😆

I agree that it's probably not perfect but I too can't find a better icon quickly so let's keep the bed icon for now - we can always change it later.
So please open a PR! :pray:

Added in #546.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v3.3 to before v5.4.134",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=a20cbdeffce247a2b6fb83cd8d22433994068565",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=d9213d4f372d30b5bc4d921795d6bed0c0e3eebf"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "a20cbdeffce247a2b6fb83cd8d22433994068565",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "d9213d4f372d30b5bc4d921795d6bed0c0e3eebf",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "memory: fsl_ifc: fix leak of IO mapping on probe failure\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.4.134 by commit d9213d4f372d30b5bc4d921795d6bed0c0e3eebf, it was introduced in version v3.3 by commit a20cbdeffce247a2b6fb83cd8d22433994068565. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001325


Data lineage system for tracking changes to your data

Introduces a Dataset class that handles the state of data (still a list of Example instances).
A Dataset has a commit_hash and some operations.
Each time your data changes (either by adding examples from new annotations, or by running a recon operation) the commit_hash changes.
This provides a sort of caching mechanism in which all of your data can be saved and loaded from disk and you can track changes based on your current data commit.
This gives you a view into the lineage of your data over time inside of recon.

Codecov Report

Merging #1 into master will decrease coverage by 11.12%.
The diff coverage is 77.72%.

@@             Coverage Diff             @@
##           master       #1       +/-   ##
===========================================
- Coverage   93.17%   82.04%   -11.13%
===========================================
  Files          10       14        +4
  Lines         410      852      +442
===========================================
+ Hits          382      699      +317
- Misses         28      153      +125

Impacted Files
Coverage Δ

recon/recognizer.py
59.57% <20.83%> (-40.43%)
:arrow_down:

recon/tokenization.py
61.06% <61.06%> (ø)

recon/corpus.py
73.52% <64.70%> (-23.03%)
:arrow_down:

recon/store.py
64.86% <64.86%> (ø)

recon/corrections.py
74.28% <76.00%> (-2.86%)
:arrow_down:

recon/loaders.py
82.35% <80.00%> (+7.35%)
:arrow_up:

recon/operations.py
81.01% <81.01%> (ø)

recon/dataset.py
87.20% <87.20%> (ø)

recon/registry.py
92.30% <92.30%> (ø)

recon/insights.py
88.88% <93.54%> (+0.17%)
:arrow_up:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb7f42d...c58468d. Read the comment docs.