Sketchy won't upload screenshots to Scumblr

I have Scumblr running in socket via unicorn and nginx, when I attempt to generate a screenshot with sketchy it sends the request through sidekiq fine. The issue is its not posting the screenshot back to sketchy. Logs look fine but at https://sketchy/api/v1.0/capture I get:

I tried changing /config/environment/development.rb to the port scumblr is running on in nginx and I get:

It looks like the callback url that scumblr is generating is "localhost". Are sketchy and scumblr running on the same host, and if so, is scumblr listening on the loopback address?

They are both on the same host, like I said though scumblr is running on a unix socket through unicorn, and an upstream in nginx. I don't know how I could get it to also listen on the localhost socket with the way its setup.

Let's make sure that what this log is indicating is correct configuration for your setup. Sketchy is available via HTTPS on port 8000? Scumblr is running on the same host and is available (via nginx) over HTTP on  port 80? A couple things to double check:

You can visit scumblr via HTTP on port 80 on the sketchy host and it loads ok

You can retrieve the image/scrape via the URLs sketchy is providing. (Replace the <IP_ADDRESS> with the hosts IP if you're trying to download from a different system)
https://<IP_ADDRESS>:8000/files/www.pcworld.com_7.txt
https://<IP_ADDRESS>:8000/files/www.pcworld.com_7.png

Are you seeing the connect attempt from sketch in the scumblr log (/logs/<environment.log>)? Are there any error messages?

Sketchy is working perfectly, its also running through nginx via ssl on port 443 in config/initializers/scumblr.rb its told 443.
config/initializers/scumblr.rb:
Scumblr::Application.configure do
  # Should Scumblr automatically generate screenshots for new results
   config.sketchy_url = "https://localhost:443/api/v1.0/capture"
   config.sketchy_use_ssl = true  # Does sketchy use ssl?
   config.sketchy_verify_ssl = false # Should scumblr verify sketchy's cert
   config.sketchy_tag_status_code = true # Add a tag indicating last status code sketchy received
  # config.sketchy_access_token = "" # Access token for sketchy

Yes, I can login, run searches, etc.
I can download the images and scrapes if I use 443 or HTTP and port 8000 but not with HTTPS and 8000.
I see POST with multiple retries in log/development.log
log/development.log:

Started POST "/results/20/update_screenshot" for ::1 at 2015-02-10 16:42:28 -0700
Processing by ResultsController#update_screenshot as TEXT
  Parameters: {"job_status"=>"COMPLETED", "retry"=>0, "sketch_url"=>"https://<IP_ADDRESS>:8000/files/www.pcworld.com_13.png", "capture_status"=>"LOCAL_CAPTURES_CREATED", "url"=>"http://www.pcworld.com/article/2599460/netflix-open-sources-internal-threat-monitoring-tools.html", "created_at"=>"2015-02-10 16:42:11.303504", "modified_at"=>"2015-02-10 16:42:27.919589", "html_url"=>"https://<IP_ADDRESS>:8000/files/www.pcworld.com_13.html", "scrape_url"=>"https://<IP_ADDRESS>:8000/files/www.pcworld.com_13.txt", "callback"=>"http://localhost:3000/results/20/update_screenshot", "url_response_code"=>200, "id"=>"20", "result"=>{"id"=>"20", "url"=>"http://www.pcworld.com/article/2599460/netflix-open-sources-internal-threat-monitoring-tools.html", "created_at"=>"2015-02-10 16:42:11.303504"}}
  Result Load (0.1ms)  SELECT "results".* FROM "results" WHERE "results"."id" = ? LIMIT 1  [["id", "20"]]
  ResultFlag Load (0.1ms)  SELECT "result_flags".* FROM "result_flags" WHERE "result_flags"."result_id" IN (20)
Error adding screenshot
SSL_connect SYSCALL returned=5 errno=0 state=unknown state
["/usr/lib/ruby/1.9.1/net/http.rb:800:in `connect'", "/usr/lib/ruby/1.9.1/net/http.rb:800:in `block in connect'", "/usr/lib/ruby/1.9.1/timeout.rb:55:in `timeout'", "/usr/lib/ruby/1.9.1/timeout.rb:100:in `timeout'", "/usr/lib/ruby/1.9.1/net/http.rb:800:in `connect'", "/usr/lib/ruby/1.9.1/net/http.rb:756:in `do_start'", "/usr/lib/ruby/1.9.1/net/http.rb:745:in `start'", "/usr/lib/ruby/1.9.1/open-uri.rb:306:in `open_http'", "/usr/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open'", "/usr/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'", "/usr/lib/ruby/1.9.1/open-uri.rb:201:in `catch'", "/usr/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'", "/usr/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'", "/usr/lib/ruby/1.9.1/open-uri.rb:677:in `open'", "/usr/lib/ruby/1.9.1/open-uri.rb:29:in `open'", "/home/rails/Scumblr/app/controllers/results_controller.rb:454:in `update_screenshot'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/abstract_controller/base.rb:189:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/rendering.rb:10:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/abstract_controller/callbacks.rb:18:in `block in process_action'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/callbacks.rb:453:in `_run__4399630651260003070__process_action__callbacks'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/callbacks.rb:80:in `run_callbacks'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/abstract_controller/callbacks.rb:17:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/rescue.rb:29:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/notifications.rb:159:in `block in instrument'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/notifications/instrumenter.rb:20:in `instrument'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/notifications.rb:159:in `instrument'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/instrumentation.rb:30:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'", "/var/lib/gems/1.9.1/gems/activerecord-4.0.9/lib/active_record/railties/controller_runtime.rb:18:in `process_action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/abstract_controller/base.rb:136:in `process'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/abstract_controller/rendering.rb:44:in `process'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal.rb:195:in `dispatch'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_controller/metal.rb:231:in `block in action'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/routing/route_set.rb:82:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/routing/route_set.rb:50:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/journey/router.rb:71:in `block in call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/journey/router.rb:59:in `each'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/journey/router.rb:59:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/routing/route_set.rb:676:in `call'", "/var/lib/gems/1.9.1/gems/bullet-4.13.1/lib/bullet/rack.rb:12:in `call'", "/var/lib/gems/1.9.1/gems/meta_request-0.3.4/lib/meta_request/middlewares/app_request_handler.rb:13:in `call'", "/var/lib/gems/1.9.1/gems/meta_request-0.3.4/lib/meta_request/middlewares/meta_request_handler.rb:13:in `call'", "/var/lib/gems/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:35:in `block in call'", "/var/lib/gems/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:34:in `catch'", "/var/lib/gems/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb:34:in `call'", "/usr/lib/ruby/vendor_ruby/rack/etag.rb:23:in `call'", "/usr/lib/ruby/vendor_ruby/rack/conditionalget.rb:35:in `call'", "/usr/lib/ruby/vendor_ruby/rack/head.rb:11:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/params_parser.rb:27:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/flash.rb:241:in `call'", "/usr/lib/ruby/vendor_ruby/rack/session/abstract/id.rb:225:in `context'", "/usr/lib/ruby/vendor_ruby/rack/session/abstract/id.rb:220:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/cookies.rb:486:in `call'", "/var/lib/gems/1.9.1/gems/activerecord-4.0.9/lib/active_record/query_cache.rb:36:in `call'", "/var/lib/gems/1.9.1/gems/activerecord-4.0.9/lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/callbacks.rb:373:in `_run__3549210540477604604__call__callbacks'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/callbacks.rb:80:in `run_callbacks'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/callbacks.rb:27:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/reloader.rb:64:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'", "/var/lib/gems/1.9.1/gems/better_errors-1.1.0/lib/better_errors/middleware.rb:84:in `protected_app_call'", "/var/lib/gems/1.9.1/gems/better_errors-1.1.0/lib/better_errors/middleware.rb:79:in `better_errors_call'", "/var/lib/gems/1.9.1/gems/better_errors-1.1.0/lib/better_errors/middleware.rb:56:in `call'", "/var/lib/gems/1.9.1/gems/rack-contrib-1.1.0/lib/rack/contrib/response_headers.rb:17:in `call'", "/var/lib/gems/1.9.1/gems/meta_request-0.3.4/lib/meta_request/middlewares/headers.rb:16:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/rack/logger.rb:38:in `call_app'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/rack/logger.rb:20:in `block in call'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/tagged_logging.rb:68:in `block in tagged'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/tagged_logging.rb:26:in `tagged'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/tagged_logging.rb:68:in `tagged'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/rack/logger.rb:20:in `call'", "/var/lib/gems/1.9.1/gems/quiet_assets-1.0.3/lib/quiet_assets.rb:23:in `call_with_quiet_assets'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/request_id.rb:21:in `call'", "/usr/lib/ruby/vendor_ruby/rack/methodoverride.rb:21:in `call'", "/usr/lib/ruby/vendor_ruby/rack/runtime.rb:17:in `call'", "/var/lib/gems/1.9.1/gems/activesupport-4.0.9/lib/active_support/cache/strategy/local_cache.rb:83:in `call'", "/usr/lib/ruby/vendor_ruby/rack/lock.rb:17:in `call'", "/var/lib/gems/1.9.1/gems/actionpack-4.0.9/lib/action_dispatch/middleware/static.rb:64:in `call'", "/usr/lib/ruby/vendor_ruby/rack/sendfile.rb:112:in `call'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/engine.rb:511:in `call'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/application.rb:97:in `call'", "/var/lib/gems/1.9.1/gems/railties-4.0.9/lib/rails/railtie/configurable.rb:30:in `method_missing'", "/usr/lib/ruby/vendor_ruby/unicorn/http_server.rb:580:in `process_client'", "/usr/lib/ruby/vendor_ruby/unicorn/http_server.rb:660:in `worker_loop'", "/usr/lib/ruby/vendor_ruby/unicorn/http_server.rb:527:in `spawn_missing_workers'", "/usr/lib/ruby/vendor_ruby/unicorn/http_server.rb:538:in `maintain_worker_count'", "/usr/lib/ruby/vendor_ruby/unicorn/http_server.rb:303:in `join'", "/usr/bin/unicorn_rails:209:in `<main>'"]

I think Sketchy may be misconfigured. It looks to me like Sketchy is telling Scumblr to download the images from https over port 8000. If Sketchy is expecting HTTP on port 8000, this would be the issue. I think you may need to configure Sketchy to return the image/scrape URLs as https port 443, or http (no "s") port 8000.

After reverting anything in sketchy's config-default.py where it said to change to ssl or https if using nginx and ssl back to the defaults of http its working. Thank you.

I appear to be having the same issue.  Are you still running SSL for sketchy even though you left the config-default.py as http?

No as I said I switched back to regular http since it wouldn't work no matter what I tried.

I had it working at one point with https.  If I can figure it out again, I'll let you know.

I got it working - let me document my solution and I'll post it.

I am also having a similar issue. Sketchy receives the instructions from Scumblr perfectly and creates the screen shots. Scumblr just never receives the image back from Sketchy...
Scumblr is available on http port 3000
Sketchy on http 8000.
{
"callback": "http://localhost:3000/results/115/update_screenshot",
"capture_status": "CALLBACK_SUCCEEDED",
"created_at": "2015-05-04 14:55:05.485637",
"html_url": "http://<IP_ADDRESS>:8000/files/www.ebay.com_4.html",
"id": 4,
"job_status": "COMPLETED",
"modified_at": "2015-05-04 14:55:18.730099",
"retry": 0,
"scrape_url": "http://<IP_ADDRESS>:8000/files/www.ebay.com_4.txt",
"sketch_url": "http://<IP_ADDRESS>:8000/files/www.ebay.com_4.png",
"url": "http://www.ebay.com/itm/Honda-Other-/221757996972",
"url_response_code": 200
},
Looking forward to your post @jwilczek

@selovitz,
Is Sketchy running on HTTP or HTTPS on port 8000? Can you post a snippet from your scumblr log for when the callback is being made?

Hi,
Make this change to your config-default.py in Sketchy from:
Line 27
BASE_URL = 'http://%s' % os.getenv('host', '<IP_ADDRESS>:8000')

TO
Line 27
BASE_URL = 'https://%s' % os.getenv('host', '<IP_ADDRESS>:8000')

The BASE_URL needs the correct schema.

Closing as resolved. If you're still having issues please feel reopen.

I have set scumblr and sketchy on two different machines. I am attempting to change the callback address sent to sketchy and I can't seem to find the location. The callback looks like this.
"callback": "http://localhost:3000/results/24/update_screenshot",

I need it to be my IP address vs localhost. Any help would be appreciated.

This should be set in you the appropriate environment file (config/environments/.rb) with the settings below:
Rails.application.routes.default_url_options[:host] = "scumblr.com" # Scumblr host and port if non-standard
Rails.application.routes.default_url_options[:protocol] = "https" # Scumblr protocol

ahoernecke, thanks. I found that if I don't restart sidekiq after refreshing the default_url_options, sidekiq will continue to send as original url. Figured I'd put this up if someone has the same problem, I changed that for an hour before restarting the whole box and realizing I should have restarted sidekiq

Thanks for the additional information. That's a good point, the sidekiq processes/threads won't pick up the new config without a restart.
Thanks!


fix(build-cli): Add baseline version and normalize JSON

Updates the typetests command to add the baseline version as a property in the typetests node of package.json. The baseline version is helpful information and previously you had to parse it out of the *-previous dependency string.
This change also always appends a trailing newline when writing JSON files, which is consistent with our repo settings.

The baseline version is the previous minor/major released version?

The baseline version is the previous minor/major released version?

It's based on what's provided when the typetests command is run. The baseline version may differ based on the branch in the future.


Improve performance of large Marlowe contracts on the playground

The current implementation of Marlowe semantics written in purescript is not performant enough for large contracts. We need to assess how we can make large contracts work nicely on the playground, possibly sending things to the backend instead of executing in the browser.

I have replaced the parsing library with one which is continuation-based and it has reduced the parse time of Lars' contract from 7s to 1s. Now though we have an issue with the pretty printer stack overflowing. I am going to write a simpler pretty printer and use the format from data types in https://gitlab.com/joneshf/purty

Pretty printer improved in https://github.com/input-output-hk/plutus/pull/1832


Update vendored docker and libnetwork

This brings in a race fix from docker/docker. It also updates libnetwork
to keep it in sync with changes in the docker/docker code.
Finally, logrus to use the same fork that docker uses now.

Also sync'd logrus with docker master.

Current coverage is 56.21% (diff: 0.00%)

Merging #1684 into master will increase coverage by 0.03%

@@             master      #1684   diff @@
==========================================
  Files            89         89
  Lines         14393      14407    +14
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           8086       8099    +13
- Misses         5235       5241     +6
+ Partials       1072       1067     -5

Powered by Codecov. Last update bcedc9f...306616c

Integration test failure is affecting multiple PRs. It's tracked here: #1680.

LGTM


Cache environment in which LUE is configured and built

Cuts down the time for the C/I builds to finish.

Fixed months ago... HPX and MDSpan are cached now. Maybe more can be cached. Fix that once needed.


[Warning message] UserWarning: None of the inputs have requires_grad=True. Gradients will be None

/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:61: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(

Have you got this warning message?
Is there any problem if i just ignore this warning message?

Hi, it appears only in iteration 0 and has no effects to the training.


Forks view and Collection Component

Reuse the same web component to show a section pages or the forks section.
It will have two views: grid and table.
Mock the fork search for now

Create a section.base.element.ts (maybe an abtract? class)
Extract the function to get pages ids and to sort them into a separate implementation class (this is the same logic as the profile page) (maybe the blog feed can be just a view over the collection.)

For the card view, we can have new "read-only" document viewer component that fetches 2 levels of the document and renders it without the editor capabilities. Let's create a task for this.

[ ] Style collections
[ ] Review flash reload when switching page.
[ ] Review push missing one update when push changes


adding script to download legacysurvey

This small PR adds missing script to download the legacysurvey from NERSC through globus

@lhparker1 I've also added a small bit of additional code to regenerate the files that had an issue. The legacysurvey should all be fine now.


expose autoscaler for both webhooks

add req/limit for operator proxy webhook
double req/limit for core webhook
equate results api/watcher requests/limits cpu/mem results watcher controller perf settings
As part of either the nightlies exposing 1.14+, or an explicit switch to 1.14 if something blocks nightly propagation,
I'm exposing horizontal pod autoscalers and related deployment cpu/mem req/limit for the webhooks.
there was some results req/limit settings that needed to be made equal to ensure obtaining of those value from k8s, as we saw some instances where more mem was denied while below the mem limit
rh-pre-commit.version: 2.2.0
rh-pre-commit.check-secrets: ENABLED

/retest


Getting a 200 when getting a user's subscriptions, but a 401 when granting free products

My token generation seems to be working fine - I can successfully query for a user's subscriptions, but the same token set gets a 401 when trying to grant a free product. A bit more detail here: https://stackoverflow.com/questions/51830079/why-is-my-azure-ad-application-not-authorized-to-grant-free-products
Is there anything more to say in this documentation about permissions, etc. that are required to allow a service to work with all the documented endpoints?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a1bdee9e-2095-4423-62c5-0beb5520dfd3
Version Independent ID: fad19d6f-2fa3-ff54-925a-020fe65cadbc
Content: Manage product entitlements from a service - UWP app developer
Content Source: windows-apps-src/monetize/view-and-grant-products-from-a-service.md
Product: windows
GitHub Login: @mcleanbyron
Microsoft Alias: mcleans

That's great news, confirmed that it looks like it's working on my end. Thanks very much for looking into it.


CloudEvent should magically recognize dict's from known services

While the current implementation of the from_dict in cloud event is round trippable, the from_dict does not work with known services.
In other words, this should not break:
with event_hub_client:
    with receiver:
        for msg in receiver:
            batch = event_hub_client.create_batch()
            batch.add(EventData(f"Receiving messages..."))
            event = CloudEvent.from_dict(
                json.loads(str(msg))
            )

I would like to see us create something more intuitive than this:
event = CloudEvent.from_dict(
                json.loads(str(msg))
            )

Something like this:
event = CloudEvent.from_str(msg)

Start with a methid like:
def gimme_bytes(o : Union[SBMessage, EHMessage, QueueMessage]) -> bytes:
   ...

You can't import any SDKs, since you're inside azure core (duck typing)


Editor: Fix Paste From Clipboard make new scene pointing to original scene file.

Fix: #69672

Too many comment. You could just add one: Clear scene path when pasting as root to make it unique. or something similar.
You can reuse !paste_parent condition instead of adding a new one.

Too many comment. You could just add one: Make sure the scene path is empty, to avoid accidental references. or something similar.

You can reuse !paste_parent condition instead of adding a new one.

done

Thanks!


Sik-Ho Tang | Review: SPPNet --1st Runner Up (Object Detection), 2nd Runner Up (Image Classification) in ILSVRC 2014.

Sik-Ho Tang. Review: SPPNet —1st Runner Up (Object Detection), 2nd Runner Up (Image Classification) in ILSVRC 2014.

Overview
In this story, SPPNet is reviewed. SPPNet has introduced a new technique in CNN called

Spatial Pyramid Pooling (SPP) at the transition of convolutional layer and fully connected layer.

This is a work from Microsoft.
In ILSVRC 2014, SPPNet has got 1st Runner Up in Object Detection, 2nd Runner Up in Image Classification, and 5th Place in Localization Task. And it got 2 papers in 2014 ECCV [1] and 2015 TPAMI [2] with over 1000 and 600 citations respectively. Thus, SPPNet is one of the worth reading deep learning papers.

Dataset
Classification: Over 15 millions labeled high-resolution images with around 22,000 categories. ILSVRC uses a subset of ImageNet of around 1000 images in each of 1000 categories. In all, there are roughly 1.3M/50k/100k images are used for the training/validation/testing sets
Detection: 200 categories. 450k/20k/40k images are used for the training/validation/testing sets.

Spatial Pyramid Pooling (SPP)

Three-Level Spatial Pyramid Pooling (SPP) in SPPNet with Pyramid {4×4, 2×2, 1×1}.
Conventionally, at the transition of conv layer and FC layer, there is one single pooling layer or even no pooling layer.

In SPPNet, it suggests to have multiple pooling layers with different scales.

In the above figure, 3-level SPP is used. Suppose the conv5 layer has 256 feature maps. Then at SPP layer,

First, each feature map is pooled to become one value (grey), thus 256-d vector is formed.
Then, each feature map is pooled to have 4 values (green), and form a 4×256-d vector.
Similarly, each feature map is pooled to have 16 values (blue), and form a 16×256-d vector.
The above 3 vectors are concatenated to form a 1-d vector.
Finally, this 1-d vector is going into FC layers as usual.

With SPP, we don’t need to crop the image to fixed size, like AlexNet, before going into CNN. Any image sizes can be inputted.

Multi-Size Training

SPPNet supports any sizes due to the use of SPP.
With SPP, variable sizes are accepted as input, different sizes should be inputted into network to increase the robustness of the network during training.
However, for the effectiveness of the training process, only 224×224 and 180×180 images are used as input. Two networks, 180-network and 240-network are trained with shared parameters.
Authors replicated ZFNet [3], AlexNet [4] and Overfeat [5] with modifications as below (The number after it is the conv layer number):

Replicated Model as Baseline.

Top-5 Error Rates for SPP and Multi-Size Training.
4-level SPPNet is used here with the pyramid {6×6, 3×3, 2×2, 1×1}.
As shown above, with SPP only, the error rates have been improved for all models. With Multi-Size Training, the error rates improve further. (10-view means the 10-crop testing from [four corners + 1 center] and the corresponding horizontal flips).

Full Image Representation
As full image can also input into CNN with the use of SPP, authors compare full image input with only using 1 center crop input:

Top-1 Error Rates for Full Image Representation.
Top-1 error rates are all improved with full image as input.

Multi-View Testing
With full image support by using SPP, multi-view testing can be facilitated easily.

Authors resize the image to 6 scales: {224, 256, 300, 360, 448, 560}.
For each scale, 18 views are generated: {1 center, 4 corners, 4 on the middle of each side} and the corresponding flips. Thus, there are in total 96 views.
And 2 full-image views plus corresponding flips are also included.

Error Rates in ILSVRC 2012 (All are Single Model Results).
SPPNet using OverFeat-7 obtains 9.14/9.08% Top-5 Error Rate on validation/test set which is the only one under 10% in the table.

Comparison with State-of-the-art Approaches (Classification)
11 models of SPPNet are used in testing. The outputs are averaged to get a more accurate prediction. This is a boosting or ensemble technique used in many CNN models such as LeNet, AlexNet, ZFNet.

2nd Runner Up in Image Classification (ILSVRC 2014).
8.06% error rate is obtained. Unfortunately, VGGNet and GoogLeNet have better performance with the use of deep models. Finally, SPPNet can only got 2nd runner-up in the classification task.

SPPNet in Object Detection

Selective Search [6] is used to generate about 2k region proposals (bounding boxes) just like in R-CNN [7].
The input image goes through SPPNet using ZFNet by ONLY one time.
At the last conv layer, feature maps bounded by each region proposal is going into SPP layer then FC layer as shown below:

SPPNet for Object Detection.
Compared with R-CNN, SPPNet processes the image at conv layers for only one time while R-CNN processes the image at conv layers for 2k times since there are 2k region proposal. The image below illustrates the idea:

R-CNN (Left) and SPPet (Right).
After the FC layer for each bounding box, SVM and bounding box regressor are also needed, which is not an end-to-end learning architecture.

Comparison with State-of-the-art Approaches (Detection)
VOC 2007

VOC 2007 Results.

Some Amazing Results in VOC 2007.
In VOC 2007 as shown above, compared with R-CNN, SPPNet with 5 scales obtained higher mAP of 59.2%.
ILSVRC 2014

SPPNet got 1st Runner-Up in ILSVRC 2014 Object Detection.
In ILSVRC 2014, SPPNet obtains 35.1% mAP and got 1st runner-up in object detection task.

References

[2014 ECCV] [SPPNet] Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition.
[2015 TPAMI] [SPPNet] Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition.
[2014 ECCV] [ZFNet] Visualizing and Understanding Convolutional Networks.
[2012 NIPS] [AlexNet] ImageNet Classification with Deep Convolutional Neural Networks.
[2014 ICLR] [OverFeat] OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks.
[2013 IJCV] [Selective Search] Selective Search for Object Recognition.
[2014 CVPR] [R-CNN] Rich feature hierarchies for accurate object detection and semantic segmentation.


Having a config/constants folder for endpoints and any configuration settings

I think we could move the endpoints file and params file from lib to another folder called config or constants or settings or something along those lines.

Sure. Call it config. Make a PR if you want to. 👍


Non overriding solution

Although its cool, its slightly evil to override other modules especially core module.
Consider having two ways of imitating the library without messing up http.

This is actually a pedological project, so there are very tight constraints. I definitely agree that overriding core functionality is evil (possibly beyond evil). However, it was necessary for this project, because we wanted students to learn NodeJS. What is awesome about this particular implementation is that the student can remove the var nat = require('nat'); line from a NodeJS application, transition to another server, and the code will still work.
We could have easily written our own .listen() method, but it wouldn't have been entirely beneficial to the students as the continue Node development... we would be creating a crutch, if that makes sense.

This is irrelevant to the original issue, but was this an introductory course to node.js or Javascript and programming in general?
Sounds like you guys were using some cool teaching techniques (although slightly unethical in production) :D

It's a distributed application development class. It's not really "teaching" any programming. The students have already taken at least 3 programming courses by the time they reach us.
Yeah, we agree that it's not good in production...it's mostly for teaching purposes.
Sent from my iPhone
On Oct 16, 2011, at 10:21 PM<EMAIL_ADDRESS>wrote:

This is irrelevant to the original issue, but was this an introductory course to node.js or Javascript and programming in general?
Sounds like you guys were using some cool teaching techniques (although slightly unethical in production) :D
--
Reply to this email directly or view it on GitHub:
https://github.com/sethvargo/node-nat/issues/1#issuecomment-2424371


Multiple issues on OSX

There are a few issues on OSX that all relate to multiprocessing module.
There are no outputs emitted from analyzer subprocesses.
The store command fails.

fixed by #4076


Misc cleanups

What was wrong?

A bunch of isolated commits
To-Do

[x] Clean up commit history

[ ] Add entry to the release notes

Cute Animal Picture

Commits are mostly unrelated, so could be reviewed one by one.


add planned projects to time series chart

The IL Clean Jobs folks think showing planned projects to the time series chart would be very helpful.
For each dataset, we can use the following info:

EIA: Planned Operation Month, Planned Operation Year
IL ABP: Scheduled Energization Date
IL SFA: Scheduled Energization Deadline (though 97% are blank)

I'll email IPA about the IL SFA for all date as well

started looking at this data and the ABP planned data has a significant number of projects with a scheduled energization date in the past, even going back to Mar 2020. I will email IPA about this as well. Not sure how it should be displayed.

Emailed with the IPA and got some more info on this:
Q: We'd like to add planned projects to the time series chart of installed MW over time, but I noticed that the Report 2 spreadsheet for ABP has a significant number of projects with a "Scheduled Energization Date" in the past - even as far back as Mar 2020. Is it possible to get an updated date for these projects? How should we interpret them? Here's a chart of that data for reference.
A: The Scheduled Energization Date is merely a date based on when the ICC approved the project. Originally DG projects were given one year, and community solar projects eighteen months, that was later updated to eighteen months for DG and two years for community solar. Projects can (and often do) request extensions to those dates, I’m not sure if that original data field is updated, or if the extension dates are stored separately.
Q: The IL SFA Report 2 has a "Scheduled Energization Deadline" column, but 97% of them have no date given (compared to 100% for ABP). Would it be possible to add those dates or infer them from something else?
A: That is odd that the ILSFA Report 2 does not have the Scheduled Energization Deadline on most records we’ll need to look into that.

update from IPA:

I wanted to provide you an update on our Reports. The Illinois Solar for All report has been updated and should reflect the Scheduled Energization Deadline field accurately now. We are currently working with our Illinois Shines/ABP team to review that report and correct fields where applicable. Let us know if you have any additional questions and we will update you with more information on the Illinois Shines/ABP report when available.


Should OG Image Templates live in includes dir?

Currently the .og.* templates can live anywhere – helps to keep the templates near where they might be needed. Plugin could also use the includes per default, would there be a benefit?

Let's keep this, only downside is additional config and the eleventyConfig.ignores.add. But still nice to have the templates near their pages. OG image templates can still be placed into _includes to whom this feels better.


Defaultsslcertargtests

Description
Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Feature # (details)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration. Is it a breaking change which will impact consuming tool(s)?

[ ] Test A
[ ] Test B

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

-28 of 52 (46.15%)  changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.7%) to 81.22%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

api/v1alpha1/zz_generated.deepcopy.go
1
29
3.45%

Files with Coverage Reduction
New Missed Lines
%

pkg/controller/ingress/concurrency_watchdog.go
1
86.41%

Totals

Change from base Build<PHONE_NUMBER>:
-0.7%

Covered Lines:
2556

Relevant Lines:
3147

💛  - Coveralls


Adding repodb file

@andrewpaliga I think this is alright

@EiNSTeiN-

@girasquid for review too

:+1:

:+1:


[SPIKE] Schools autocomplete with large number of schools

What?
https://trello.com/c/7OGaCU6j/1327-spike-1day-autocomplete-for-schools-lookups
This PR is the output for the above spike - experimenting with providing a large list of schools to an autocomplete for UR.
How?
1. Switching to a standalone 'autocomplete' text input

The Accessible Autocomplete can be used either to progressively enhance a select input, or as a standalone text input. Mostly we use it as an enhancement, since this makes the non-JS fallback easy (it's just a select). However, this is not feasible with ~40k possible results for two (or more) reasons:

The non-JS fallback of a select with ~40k records is... no.
The autocomplete performance searching across this number of records is not good enough

2. Doing the filtering ourselves

The autocomplete normally takes a simple array of strings as it's source. However, you can also provide it with a function. This function is passed two arguments:

query (string) - i.e. the search term from the user
populateResults (function) - i.e. a callback which expects to be passed an array of results to render

This way, the filtering and sorting of results is completely down to the developer to handle (either on the server or client-side) and we can improve the performance.

3. Actually improving the performance

Without a database, this is a little tricky. In order to achieve a performant text search, we need some kind of index.
Enter https://lunrjs.com/ - a package for creating simple search indexes on documents (json)
Creating the search index every time we need it would be slow, so here I'm generating it once when the app loads and saving it.
Then the autocomplete can use this pre-built search index to quickly return matches

4. Hooking it all up

The schools autocomplete makes a request to our server for the search index when the page loads
When users enter a search term we call a (debounced) function which queries this index for our search term
When the result is selected, a hidden field is populated with the UUID, and the name is shown in the autocomplete input
The UUID is sent to the server to match the correct school (there are multiple schools with the same name so we can't just submit the name)

And now?

[ ] This doesn't solve the non-JS issue, this should be thought about in a separate spike. Make a ticket.
[ ] This doesn't use the actual schools list. Generate the correct json and include.
[ ] The default text 'No results found' is misleading on occasions when we're actually still searching. Make it say 'Loading...' or something

 Spike [1day] autocomplete for schools lookups


fix: optimize inscription backfill indexes

:tada: This PR is included in version 1.0.0-brc-20.7 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[experimental] Add an NFT example module with corresponding demo comm…

…ands
Note that all the modules are currently published under 0x1, which is not going to be the case long term (we'll separate them to different packages)

Motivation
(Write your motivation for proposed changes here.)
Have you read the Contributing Guidelines on pull requests?
(Write your answer here.)
Test Plan
(Share your test plan here. If you changed code, please provide us with clear instructions for verifying that your changes work.)
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/diem/diem/tree/main/developers.diem.com, and link to your PR here.)
If targeting a release branch, please fill the below out as well

Justification and breaking nature (who does it affect? validators, full nodes, tooling, operators, AOS, etc.)
Comprehensive test results that demonstrate the fix working and not breaking existing workflows.
Why we must have it for V1 launch.
What workarounds and alternative we have if we do not push the PR.

/land

/cancel

Rebased on latest. Could you approve and land again? Thanks!

Sorry, had to rebase again... approve?

/land

:broken_heart: Test Failed - ci-test

/land

:broken_heart: Test Failed - ci-test

/land

:broken_heart: Test Failed - ci-test

I commented out the newly-reintroduced move prover testing of experimental since it doesn't pick up the address assignments from experimental Move.toml.
Will be happy to bring it back once I figure out how to do it right! :)


Support for the Social Accounts Auth?

The way accounts-google works with meteor is it creates a popup window that it passes auth back through, and electron doesn't create a popup from the social buttons so you actually can't auth from the social networks.
I have seen an implementation using Auth0 that inlines the google auth, but I went down this rabbit hole hoping to just tie into my existing Meteor Google Auth that I've currently been using with an existing project.
(I also tried to using the loginStye: redirect instead of the popup which again works in the webapp but doesn't appear to do anything in electron and not firing any console errors)

Hey,
sorry to keep you waiting, I will check that over the weekend. It is weird because until you would intentionaly block it in the new-window event it should work normally.

FYI got a rough time now and it did not have time yet to test this, but do not worry it is still on my radar. I will investigate this on tomorrow.

Ok, so I have reproduced this. Indeed something is wrong with window.open in Electron. I am checking if that is a bug I can file to Electron.
Apart from this problem, the are others. Next problem will be a little bit more complicated as google wants a redirect url to be provided without any wildcards. Meteor-desktop starts the app on a random port to prevent conflicts with other meteor-desktop apps. The solution would be to provide several urls with all possible ports but that seems to be an overkill.
Oh and the accounts-google is by default passing the meteor server url not the http://<IP_ADDRESS>: on which meteor-desktop runs your app so that is another problem.
So... basically, meteor-desktop env resembles cordova. Given that meteor guide recommends using Meteor.loginWith<externalService> in Cordova I would advise you to do the same. Hope that is something you can consider.
Here is a gist that might be helpful: https://gist.github.com/yhagio/dcb24b7a74bd21d02838
I will make this problem something I will consider to fix in 1.x but for now, I will not be able to do it as this would require some architectural changes.

@wojtkowiak I am using social login just like that gist but it does not work. In your last post are you referring to the gist as an alternative way to get social login to work with meteor-desktop? Or is social login not possible at all right now with meteor-desktop?
I have disabled social login on desktop app but if there is an alternative way forward to get it working please let me know. I get the impression that it is currently not possible and will be fixed closer to 1.0 release. Thanks

@sferoze well Meteor.loginWith<externalService> is normally supported in meteor so it should work. You can first try to get a working version in the browser normally running your app.
However I am not sure if the gist is 100% correct. Unfortunately I did not have to use any external login service yet in my desktop projects.  Are you getting any concrete error?
Have you tried to look for an another example of using that? You can also ask on the meteor forum for help with that. I can probably also help but I would need a small project prepared for working on that just to save some time on my side.

@wojtkowiak The social accounts package uses window.open to show a popup window. I get an error saying the popup window does not exist after trying to use Meteor.loginWithGoogle
I think there is an issue with electron and the popup window code it tries to use. I am just trying to use the standard meteor social login.
I will prepare a small test repo with just social login and post back here.

Well I did some digging and this seems to be much more complicated than I thought.
Actually what you probably need is a mix between:
https://www.npmjs.com/package/electron-google-oauth
https://github.com/sujith3g/meteor-g-plus
I would be very interested in providing that as a plugin to meteor-desktop but unfortunately I would not be able to collaborate much on that in the next two months.

@sferoze @iamkevingreen did you manage to get through this issue or is this problem still standing?

Any improvements in this ?

Hi @wojtkowiak ,
I'm working on this now to ad Azure Ad login to my app, but using the redirect style rather than pop-up. I have a woking setup in browser, but on Electron when I try and login nothing appears to happen on the app.
I see the following in the logs when I hit Meteor.loginWith
debug: [main] will-navigate event to https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=bc222e34-89d1-412f-a069-7691d1800827&response_type=code&redirect_uri=http%3A%2F%2F<IP_ADDRESS>%3A3000%2F_oauth%2FazureAd&scope=offline_access user.read&response_mode=query&state=eyJsb2dpblN0eWxlIjoicmVkaXJlY3QiLCJjcmVkZW50aWFsVG9rZW4iOiJXdzlsNTl3a1BqTTNzbHFiR3RqMVdfX3NrX003aUVfTDVYUUFZRzFOanNzIiwiaXNDb3Jkb3ZhIjpmYWxzZSwicmVkaXJlY3RVcmwiOiJodHRwOi8vbG9jYWxob3N0OjMwMDAvbG9nZ2luZ0luIn0%3D&prompt=login&login_hint=alex.chapman%40candivue.com, assuming that this is HCP refresh
I ran a packet capture on my machine and this doesn't appear to trigger anything on the wire at all.
One thing to note is that I'm running a much older version of Meteor-desktop - 0.6.2 because there is a Chromium bug in the newer versions that I need to avoid.
Before I go down the rabbit hole of trying to fix that, I wondered if you had any theories as to what might be happening here?
Thanks!

The issue described above went away after upgrading to meteor-desktop 2.2.5.
I've now got the issue of invalid redirectUri but will work on figuring that out next


[Issue] User Management: Update User Prisma model

Subtask of user story: User Management https://github.com/usdigitalresponse/cpf-reporter/issues/81
DOD: The User Prisma model is updated to only implement direct relations and make required foreign keys non-nullable.
Blocked by issues: (None)
Blocks these issues:

https://github.com/usdigitalresponse/cpf-reporter/issues/101
https://github.com/usdigitalresponse/cpf-reporter/issues/102

Implementation details:

Define a User Prisma model that implements the following schema:

Columns:

id: Int, pk
email: string, valid email address, required
name: string, required
agencyId: int, fk to Agency.id, required
createdAt: DateTime with timezone, default now(), required
updatedAt: DateTime with timezone, default now(), required
role: value of Role enum, required
isActive: boolean, default true, required

Relations:

agency:

type Agency
via User.agencyId referencing Agency.id

certified:

type []ReportingPeriod (to-many)
via ReportingPeriod.certifiedById referencing User.id

uploaded:

type []Upload (to-many)
via Upload.uploadedById referencing User.id

Seed development fixtures for each user role (and dependencies thereof, e.g. Agency and Organization) in scripts/seed.ts

Fixtures that can successfully seed a development environment will suffice in lieu of unit tests for this issue.

@TylerHendrickson for valid email validation, do we want to enforce this in the backend and database layer or is it sufficient to handle on the front-end?


Don't log errors on picking up XP orbs

It's extremely common to pick up XP orbs in Minecraft. Currently, an error is thrown every time one is picked up. These don't hurt the bot in any way, but they are extremely spammy in the console.
using hideErrors: true fixes this, but can pose other issues by hiding other important issues from other plugins or bot code.

I believe I fixed this with #36


Tokenize & Authorize gift in BGE

Critical Changes
Changes
Issues Closed
Community Ideas Delivered
New Metadata
Deleted Metadata

@sokosaurus Added suggested changes, and other than that, it all looks good!


Tradfri problem.

I can't get to get tradfri working. I need to put my Security Code in, at the HA panel ( homepage) and it doesn't do nothing, when i check the error log i am see this message.
2017-11-06 23:16:36 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last):   File "/usr/lib/python3.6/asyncio/tasks.py", line 179, in _step     result = coro.send(None)   File "/usr/lib/python3.6/site-packages/homeassistant/components/tradfri.py", line 66, in configuration_callback     psk = yield from api_factory.generate_psk(callback_data.get('key'))   File "/usr/lib/python3.6/site-packages/pytradfri/api/aiocoap_api.py", line 193, in generate_psk     self._psk = yield from self.request(command)   File "/usr/lib/python3.6/site-packages/pytradfri/api/aiocoap_api.py", line 149, in request     result = yield from self._execute(api_commands)   File "/usr/lib/python3.6/site-packages/pytradfri/api/aiocoap_api.py", line 141, in _execute     api_command.result = _process_output(res, parse_json)   File "/usr/lib/python3.6/site-packages/pytradfri/command.py", line 71, in result     self._result = self._process_result(value)   File "/usr/lib/python3.6/site-packages/pytradfri/gateway.py", line 28, in process_result     return result[ATTR_PSK] TypeError: 'NoneType' object is not subscriptable

Please ask on Home-Assistant repository. Thanks for reporting


Allow running bitwarden2hashcat against a specific data.json file

It would be great if this tool would allow to specify the path to a specific file (like in case of a pentester who gathered some .json files that they want to "convert"):
python bitwarden2hashcat.py /tmp/2020_08_13_0912/client1337/data.json

Several *2hashcat and *2john tools allow specifying a specific file and it will be converted to a hash. This would be a great addition thx

Actually, I'm on vacation right now.
I'll get back to this in a week or two.
Have a good time!

I implemented this now with e455470.
Also, I changed the licence to BSD-3 to extend compatibility to hashcat and JTR.
I am very happy to have had the opportunity to contribute something to hashcat!
Feel free, to open another issue if neccessary.

Wow.
Thank you so much.
This update is great.
Looking at the code, I just think there might be 1 single instance that doesn't work yet (but it's of course not a big deal):
what if the user tries to run this:
bitwarden2hashcat.py /tmp/a.json /home/philsmd/Downloads/data.json /data.json
most other *2hashcat or *2john tools allow this scenario.
This is of course a new feature request if it's currently not working... I just wanted to mention it here, because as my very tiny python understanding goes, it currently doesn't/shouldn't work with the update, right ?

Thank you again for this great tool and all your help/support also with explaining the algorithm etc.
Well done.

I am still very glad that I can/could help.
You're welcome.
I also implemented your suggestion, since it was just a small change and little of work.
Am looking forward to helping again!


Adding events for Payment Reject

Created from #4525
This covers adding events for Payment Reject operation
This event should include the payment data similar to PaymentCancel
It should also include any metadata for the event
e.g reason for payment rejection, error codes, rejection source etc

Hi @lsampras can I work on this issue?

Hey @aadityasinha-dotcom , this is already claimed by @subhajit20,
You can check for similar issues in the tracking issue #4525

@lsampras sure np!!

hey @lsampras can I take this issue?

@swangi-kumari This issue has been inactive for a long time now. I want to take this up. Please assign this to me. Thanks.

Hey @subhajit20 Are you still working on this issue?

Hey, I would like to work on this issue. The other issue I was assigned is approved and only left to be merged..

Hey @AnishKacham Assigned to you, as this was inactive for a while.

@lsampras How do I know what all metadata to send in the audit event. I understand that PaymentCancel audit event is given as an example.
Is there any place where the different fields of PaymentData struct are documented to understand which fields can go as a metadata for a PaymentReject?

Hey @AnishKacham,
This is a bit subjective the goal is to capture all relevent context available with that change/transition,
you can start off with an empty event and reviewers can point out any fields that need to be added or removed from the event...


EPT and LAPIC testing

Build and run EPT unit tests
Write, build, and run LAPIC unit tests
Bug fixes for conversion and signedness warnings

I might have missed it, but can we turn some of the Travis CI tests back on as part of the PR?

Also, we should be able to get Windows working as well now that Windows is working on the main hypervisor side of things.


Add implementation of fetch api

See: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

Thanks! For now, I've moved Fetch to its own package. I'll have a look later so we can decide the easiest way to make the interaction with user code :1+:


[TEST] Upgrade test always failed in master-head

What's the test to develop? Please describe
Like https://ci.longhorn.io/job/public/job/master/job/sles/job/amd64/job/longhorn-upgrade-tests-sles-amd64/458/ or https://ci.longhorn.io/job/public/job/master/job/sles/job/arm64/job/longhorn-2-stage-upgrade-tests-sles-arm64/167/
Probably related to https://github.com/longhorn/longhorn-tests/pull/1277.
In that PR, the wait_for_all_instance_manager_running is changed to check instance_manager instead of instance_manager-e/r. But in upgrade test, the previous version of Longhorn is installed first and it still uses instance_manager-e/r.
Describe the items of the test development (DoD, definition of done) you'd like

Please use a task list for items on a separate line with a clickable checkbox https://docs.github.com/en/issues/tracking-your-work-with-issues/about-task-lists

[ ] item 1

Additional context
Add any other context or screenshots about the test request here.

cc @c3y1huang

Fixed in https://github.com/longhorn/longhorn-manager/pull/1920.


docs: (IAC-1132) Add License Update Doc

Changes
Adds a section in the troubleshooting guide on how to update you SAS Viya platform deployment license using viya4-deployment.

Small update after user suggestion.


Concerning making sure about getting into the welcome page

Hi.
It seems the code (insta_feed_scraper.py) entered into infinite loop when I tried to run it, and I found it was likely due to the part below (line 298).
            # Make sure that we get into the welcome page
            while True:
                driver.get(main_url)
                time.sleep(3)
                try:
                    if driver.find_element_by_xpath("/html/body/div[1]/section/main/div/header/section/div[1]/h2"):
                        print(colored("\n[SUCCESS]: Got into the user page. \n", "green"))
                        break
                except NoSuchElementException:
                    pass

The code tries to find "/html/body/div[1]/section/main/div/header/section/div[1]/h2" but it doesn't exist.
Could you tell me what element the code is supposed to find?
My guess is that it tries to find the user (account) name part. If my guess is correct, should the "h2" in the last be replaced with "h1"?
Thank you in advance.

I downloaded the code again, and it seems work. I'll keep trying find out what was wrong.
Thank you anyway.


Changing our release and hotfix system

Currently, all releases are done from master, and all hotfixes to release issues are also submitted to master. Brief history of the problems that we are encountering with our current release system:

We had a major issue with Dagger crashes in 2.6.5, which was solved by #1062
#1062 was released via 2.6.6
2.6.6 fixed the Dagger crash from 2.6.5, but also included other changes that caused a different type of crash, with an even higher crash rate
We wanted to roll back to 2.6.5 while cherry-picking the Dagger fixes
This turned out to be extremely difficult because the Dagger fixes built on other code changes that were submitted to master between 2.6.5 and 2.6.6.

In the future, I'm thinking that we might want to change how we do hotfix releases. My proposed new system:

Every major release (e.g. 2.6), I will pull from master and create a new branch (e.g. 2.6.x-release)
Hotfixes to the major release must be done in the 2.6.x-release branch, not in master
Hotfix releases (e.g. 2.6.1) will be pulled from the 2.6.x-release branch. New changes/enhancements to master will not be included in the hotfix release
Release branches will be merged into master periodically (so master is up to date with hotfixes made to release branches)
Hotfixes can be released to production after being tested in beta for a few days. Major releases on the other hand should be tested in beta for a much longer time (e.g. 2 weeks?)

I believe that this will allow us to make hotfixes more easily and independently of master, and reduce the headaches that we are facing currently. Given that major releases are expected to be the least stable, they should also be tested the most vigorously.

Workflow now in use and documented on wiki.


 Do you think we can highlight the this emoji reference:
https://github.com/dannyfritz/commit-message-emoji

I like that idea, go ahead with it.

Closing issue per merging of #6 . 🎊 🙏 @zerodom30


[5.0] Fix Unit Testing Facade Issues

This pull request is a fix for the problem described in #7367.

:+1:

Would it be better to clear them before calling the kernel rather than after?

Maybe. Not sure if that would be breaking for anyone?

Doing it before Kernel should be fine, however we need to reset app state to not has been bootstrapped in order for all bootstrap class to be executed again.

Didn't quite follow that comment @crynobone

See https://github.com/laravel/framework/blob/5.0/src/Illuminate/Foundation/Http/Kernel.php#L142
After the first request if we flush resolved instance and tun kernel again, none of the bootstrap class will be loaded since app has been bootstrapped before.

Ah, I'm not sure if we care about that. Probably best bootstrappers only once?

Might be just overthinking this, haha.
Another concern is how would you define mocked facade between both request?

Solved this at the HTTP handler level.


fix: pickle issue with exception

:tada: This PR is included in version 0.11.0-beta.22 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


error: 'initialize_GodotSteam_module' was not declared in this scope

Describe the bug
modules/register_module_types.gen.cpp: In function 'void initialize_modules(ModuleInitializationLevel)':
modules/register_module_types.gen.cpp:204:9: error: 'initialize_GodotSteam_module' was not declared in this scope; did you mean 'initialize_godotsteam_module'?
  204 |         initialize_GodotSteam_module(p_level);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         initialize_godotsteam_module
[100%] modules/register_module_types.gen.cpp: In function 'void uninitialize_modules(ModuleInitializationLevel)':
modules/register_module_types.gen.cpp:355:9: error: 'uninitialize_GodotSteam_module' was not declared in this scope; did you mean 'uninitialize_godotsteam_module'?
  355 |         uninitialize_GodotSteam_module(p_level);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         uninitialize_godotsteam_module

To Reproduce
Steps to reproduce the behavior:

Follow build instuctions here: https://godotsteam.com/howto/modules/#__tabbed_2_3
Build with scons custom_modules='../path/to/dir/godotsteam' tools=yes
See error

Expected behavior
No build errors
Desktop (please complete the following information):

Void Linux

Version of Godot:

Master branch
4.2 branch

Version of GodotSteam:
godot4 branch

It appears that i must have the path be godotsteam and not GodotSteam (as a straight clone is) otherwise godot generates the init names with caps. My mistake i missed that.

Hey there!  I was just reading through this then digging in the source code to see if I accidentally changed something.  Haha.

Compiles clean now, thanks!

No problem!  Let us know if you run into any other issues!


I have an issue and idk how to fix it

Describe the issue
I tried to play and got this error

Instructions to reproduce
Idk what to do
What version of LiveContainer are you using?


Figure out how to run unit tests in ec-policies that use custom function

Description
The function introduced in https://github.com/enterprise-contract/ec-cli/issues/1070 means that a policy rule could write something like this:
ec.fetch_oci_bundle(ref)

Plain OPA doesn't know anything about that function. If a policy rule in ec-policies wants to use that function, this error will occur in the tests:
rego_unsafe_var_error: var ec is unsafe

If we try to mock that in the tests by using the with keyword:
rego_type_error: with keyword target must reference existing input, data, or a function

One option could be to wrap the opa command in ec-cli.
This also raises the question of how we can mock those functions. We don't want to actually hit a registry during the unit tests, for example.
Acceptance Criteria

ec-provided custom rego functions can be used in ec-policies without
breaking tests.

Do we need ec policy test command?

Maybe? That might be nicer than just wrapping the whole opa command. It should be compatible with opa test though. So wrapping may still be involved here.

I've asked OPA maintainers about bringing back shared library support for loading plugins into OPA. This is not something that will be considered in OPA's future.
Based on this I think the best option would be to add support for running tests into ec, most likely based on this pattern.
Perhaps we can have ec opa command that would behave exactly the same as opa?


two switch cases to the same logic doesn't work

Hi,
I develop Go using goland, I noticed that using:
switch x:
case y:
case z:
logic1
default:
logic2
If I enter the switch with value y, I will jump to default behaviour (logic2) and not what I expected which is logic1, would be nice if I could use this syntax.
Thanks,
Elias

The syntax you want is
switch x {
case x, y:
   logic1
default:
  logic2
}

Unlike many projects on GitHub, the Go project does not use its bug tracker for general discussion or asking questions. We only use our bug tracker for tracking bugs and tracking proposals going through the Proposal Process.
Please see https://golang.org/wiki/Questions for good places to ask questions.

Hi,
Sorry for the misuse.
And thanks for the explanation.
Elias

You're welcome, and welcome to Go!


Homework 3

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
completed homework_3
What did you learn from the changes you have made?
Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?
Were there any challenges? If so, what issue(s) did you face? How did you overcome it?
How were these changes tested?
A reference to a related issue in your repository (if applicable)
Checklist

[x] I can confirm that my changes are working as intended

HW3 marked.


[Snyk] Upgrade express from 4.17.3 to 4.18.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade express from 4.17.3 to 4.18.0.

:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 21 days ago, on 2022-04-25.

Release notes

Package name: express

4.18.0 - 2022-04-25
Add "root" option to res.download
Allow options without filename in res.download
Deprecate string and non-integer arguments to res.status
Fix behavior of null/undefined as maxAge in res.cookie
Fix handling very large stacks of sync middleware
Ignore Object.prototype values in settings through app.set/app.get
Invoke default with same arguments as types in res.format
Support proper 205 responses using res.send
Use http-errors for res.format error
deps<EMAIL_ADDRESS>
Fix error message for json parse whitespace in strict
Fix internal error when inflated body exceeds limit
Prevent loss of async hooks context
Prevent hanging when request already read
deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>
Add priority option
Fix expires option to reject invalid dates

deps<EMAIL_ADDRESS>
Replace internal eval usage with Function constructor
Use instance methods on process to check for listeners

deps<EMAIL_ADDRESS>
Remove set content headers that break response
deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>
Prevent loss of async hooks context

deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>
Fix emitted 416 error missing headers property
Limit the headers removed for 304 response
deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>
deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>
Remove code 306
Rename 425 Unordered Collection to standard 425 Too Early

4.17.3 - 2022-02-17
deps<EMAIL_ADDRESS>
deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>
deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>

deps<EMAIL_ADDRESS>deps<EMAIL_ADDRESS>
Fix handling of __proto__ keys

pref: remove unnecessary regexp for trust proxy

    from express GitHub release notes

Commit messages

Package name: express

547fdd4 4.18.0
0b330ef bench: print latency and vary connections
158a170 build: support Node.js 18.x
29ea1b2 build: use 64-bit Node.js in AppVeyor
11a209e build: support Node.js 17.x
fd8e45c tests: mark stack overflow as long running
708ac4c Fix handling very large stacks of sync middleware
92c5ce5 deps<EMAIL_ADDRESS>8880dda examples: add missing html label associations
b91c7ff examples: use http-errors to create errors
ecaf67c docs: remove Node Security Project from security policy
99175c3 docs: fix typo in casing of HTTP
1b2e097 tests: fix typo in description
04da4aa build: use<EMAIL_ADDRESS>for Node.js 6.x
2e2d78c deps<EMAIL_ADDRESS>980d881 deps<EMAIL_ADDRESS>1df7576 deps<EMAIL_ADDRESS>32c558d deps<EMAIL_ADDRESS>a107702 Use http-errors for res.format error
5855339 Fix behavior of null/undefined as "maxAge" in res.cookie
1cc8169 deps<EMAIL_ADDRESS>9482b82 Invoke default with same arguments as types in res.format
10b9b50 examples: use updated res.download in example
03dc367 Allow options without filename in res.download

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Codecov Report

Merging #105 (a10c722) into master (22868a0) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master      #105   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines           91        91
=========================================
  Hits            91        91

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


[TLS] Cannot connect to remote db

As the title states, I have a Linode MySQL cluster and need to connect to it via TLS, it's a mandatory requirement. I have passed the certificate path and tried verifying peers, using local_cert and more but it still won't connect. I use clue/framework-x and inside my DI container, I instantiate the connection like so:
use React\MySQL\Factory as ConnectionFactory;
use React\Socket\Connector as SocketConnector;

...

ConnectionInterface::class => function (EnvironmentInterface $environment) {
      $user = $environment->getDatabaseUser();
      $password = $environment->getDatabasePassword();
      $host = $environment->getDatabaseHost();
      $name = $environment->getDatabaseName();
      $certificate = $environment->getMySqlTlsCertificatePath();
      $connector = new SocketConnector([
          'tls' => [
              'cafile' => dirname(__DIR__) . "/" . $certificate
          ]
      ]);
      $credentials = "{$user}:{$password}@{$host}/{$name}";
      $factory = new ConnectionFactory(null, $connector);

      return $factory->createLazyConnection($credentials);
}
...

The information is all correct and I can see the $credentials contains all necessary values. This is my current version but as I said, I have tried with verify_peer, verify_peer_name, local_cert, etc but still it cannot connect.
Using my local MySQL GUI, the connection with the exact same connection details and certificate is fine and so it must be something to do with the connector as far as I can understand it and the error thrown is:
[Mon May 13 17:14:56 2024] PHP Fatal error:  Uncaught TypeError: method_exists(): Argument #1 ($object_or_class) must be of type object|string, null given in /Users/<user>/<path/to/project>/vendor/react/dns/src/Query/TimeoutExecutor.php:57

Note: I edited the path to the error for this issue, in case it was confusing for anyone.

Anyway, if someone can point me to the right direction I would appreciate it because I have my .crt file and just need to connect to the database but the documentation for this and TLS isn't very clear and I have tried everything I can think of now, so any help would be great 🤷🏻.

@SimonFrings @clue any ideas?

Oh and the ConnectionInterface container implementation currently looks like this:
...
ConnectionInterface::class => function (EnvironmentInterface $environment) {
        $user = $environment->getDatabaseUser();
        $password = $environment->getDatabasePassword();
        $host = $environment->getDatabaseHost();
        $name = $environment->getDatabaseName();
        $certificate = $environment->getMySqlTlsCertificatePath();
        $connector = new SocketConnector([
            'tls' => [
                'cafile' => dirname(__DIR__).'/'.$certificate,
                'verify_peer' => true,
                'verify_peer_name' => true,
                'allow_self_signed' => false,
            ],
        ]);
        $credentials = "{$user}:{$password}@{$host}/{$name}";

        $factory = new ConnectionFactory(null, $connector);

        return $factory->createLazyConnection($credentials);
    }
...

Hey @jamesrweb, thanks for bringing this up :+1:
This is not a configuration problem on your end, it's because we don't have built in support for secure TLS connections. There's also an open feature request for this in #49 which also depends on the open ticket in Socket https://github.com/reactphp/socket/issues/89.
I can't really say when this feature will come, as our highest priority is currently to prepare ReactPHP v3 for the upcoming ReactPHP birthday. If you need this more urgently you can also reach out to us.
Let's use #49 for further discussion about this topic, so I'll close this ticket here.


Cannot copy to clipboard on Ubuntu 24.04

I'm using the v1.1.0 binary from the release section.
Here's a workaround:
$ code2prompt -o /dev/stdout my-folder/ | xclip -selection c

Issue persists in v2.0.0.

I made a workaround with ChatGPT. It's just a custom script that uses a temporary output file and uses cat to get it to the clipboard with xclip. https://chatgpt.com/share/66eadf1d-ab08-8003-8c10-385f1b4be36b
`#!/bin/bash
Create a temporary file
TMPFILE=$(mktemp)
Run code2prompt with the provided options and save the output directly to the temp file using -o option
code2prompt "$@" -o "$TMPFILE"
Output the contents of the file and pipe to xclip (clipboard)
cat "$TMPFILE" | xclip -selection clipboard
Delete the temporary file
rm "$TMPFILE"
echo "Prompt copied to clipboard."
`

Same for me. 6.8.0-45-generic #45~22.04.1-Ubuntu x86_64 x86_64 x86_64 GNU/Linux

Same issue in v2.0. (built rom source) Linux isengard 6.6.57 #1-NixOS SMP PREEMPT_DYNAMIC Thu Oct 17 13:24:38 UTC 2024 x86_64 GNU/Linux


about where in

SELECT * FROM fruit WHERE name IN (apple','banana) AND color = 'red'
how to add "red" ?
$DB->query("SELECT * FROM fruit WHERE name IN (?) AND color = ? ",array('apple','banana'));

$DB->query("SELECT * FROM fruit WHERE name IN (?) AND color = ? ", array('apple','banana', 'red'));

看来我设计时还是考虑不周。

@chainjoy, your query should be:
$DB->query("SELECT * FROM fruit WHERE name IN (?,?) AND color = ? ",array('apple','banana','red'));

In the latest version:
$DB->query("SELECT * FROM fruit WHERE name IN (:fruit) AND color = :color ",array(
    'fruit' => array('apple','banana'),
    'color' =>'red')
);


Figure out new home for current_msg_list and friends

We have a cluster of 4 variables that are kinda core to the Zulip application:

home_msg_list
narrowed_msg_list
all_msg_list
current_msg_list (always equals a pointer to home or narrowed).

As part of eliminating globals in zulip.js, we moved two of these to names like message_list.all, but I am now thinking that proposed new name was a mistake on my part, because e.g. message_list.narrowed  and message_list.current sound like bools, not an object you do lots of things with.
So, we need to pick a new home for these.  Options include (using the narrowed one since it's the longest):

narrowed_msg_list (i.e. just keep them as globals)
message_lists.narrowed
message_lists.narrowed_list
msg_lists.home_list
views.narrowed_list

Important considerations:

Length: These things show up all over the code, often on already-long lines, so super long names for them are problematic.
Clarity:  These are core pieces of Zulip and it should be clear what they refer to.
Having a place to put related functions: It'd be nice to be able to do things like message_lists.for_each_list.

Here's some random lines of code using these to help in our thinking:
home_msg_list.select_id(new_pointer, {then_scroll: true, use_closest: true});
if (event.msg_list === home_msg_list && page_params.narrow_stream === undefined) {
_.each([message_list.all, home_msg_list, message_list.narrowed], function (msg_list) {
if (current_msg_list !== home_msg_list) {
home_msg_list.clear({clear_selected_id: false});
home_msg_list.view.rerender_messages(msgs_to_rerender);

Thoughts welcome!

My initial leaning is toward a small collection class like message_pane that mainly hides the details of the four different message lists from the main code and provides a slightly higher level of abstraction for any situation where you're possibly touching multiple message lists or inquiring about multiple message lists.

Closing this, as it's super stale, we've got the message_lists module, and are in the process of removing message_lists.narrowed in #22452 to further improve things, in favor of APIs like message_lists.all_rendered_message_lists.


Update Hybrid Key Trust Pre-req (New PR for missed change)

This PR is created because a previous PR #3616 for the issue #3248 was merged and a last change was missing. based con the @nenonix  review I add the change and generates this new PR to solve totally the issue #3248.  @mypil Please review and merge !

(Replaced by #3827)


adding redis example to the readme file

This does not need to be added to the readme since it exists within the Examples folder. Most rust devs will look in the examples folder on the github.


Design release pipeline in internal repo

Description
To streamline our release workflow with Helm charts, we've introduced a new process where the appVersion in the Charts.yaml file needs to be updated for each release. To ensure the accuracy and quality of our releases, we propose implementing a manual release process where certain quality checks are conducted before merging the release PR.
Create the following GitHub-Actions:

Check if the given appVersion is the latest release version.

Required check

Run all e2e tests from lifecycle manager repo.

Setup an e2e test environment based on helm charts config.
Check which E2E Test fits for helm chart config
Reuse existing GH Actions form KLM repo

The pipelines are getting triggered if a PR gets opened which includes changes to the charts.yaml.
Reasons
No response
Acceptance Criteria

[ ] Implement above mentioned pipeline as a required job before merge in management plane charts.

Feature Testing
No response
Testing approach
No response
Attachments
No response

Test PR: https://github.com/jeremyharisch/charts-testing/pull/1


PubSub Support

I'd like to begin implementing Twitch's new PubSub system for more in-depth event data.

If anyone knows anything about Twitch's PubSub, I'm all ears. Currently trying to learn how it works. I just pushed some groundwork code that is able to send requests to PubSub successfully.

BTTV's implementation of PubSub's chat_login_moderation topic for reference: https://github.com/night/BetterTTV/blob/e4b09b8246e49bfb687c5f69178a0da17b4dc24d/src/js/chat/takeover.js#L105

Support for timeouts, bans, and unbans successfully added and tested 💃

awesome I have to look test that new feature 👍

It looks like PubSub is disconnecting after around 10 minutes. I think this is due to Pings not being sent. I will work on this now.

Looks like a 3 minute PING timer applied to pubsub has fixed the disconnect issue after 10 minutes. Pushing this fix now.

Just added support for bitsevents as well! Won't be able to test until tonight though (when my streamer goes live).

awesome :) 👍

I can confirm the bits event works 💃

Added video-playback, although can't confirm it works.

video-playback ? what data provides that event ?

From what i read, video-playback should give an event on stream up, stream down, and periodic updates of viewer count. Though I've seen some chatter of it not working/being sporadic.

ohh that sounds nice

I haven't been able to get it to work though, unfortunately.

ohh ok :)

PubSub is supported to the extent (and more so) that Twitch publicly documents. I will open this again when Twitch openly supports more topics.


Add the ability for CF to unzip an artifact that could contain 1 or more files

Currently to push an application that does not get packaged/zipped (.NET, NodeJS, Golang) I just go to the root directory of the project where the manifest.yml file is and run "cf push" without specifying a file to upload. If the file is a .jar or .war file I can specify to upload just that file. It would be nice if I could specify a packaged file (.zip, .tar) that contained my .NET, NodeJS or Golang app.
Here is an example: The CI/CD tool can create artifacts that are directories or single files and these artifacts can be passed around to different parts of the deployment project. The add-ons we wish to use require a single file to be specified for uploading, which works fine for Java apps but not apps written in .NET, NodeJS, Golang, etc. I can tell Bamboo to package up a directory into a single file and keep it as an artifact but Cloud Foundry will not accept that packaged/zipped file when doing the push. Because of this, the deployment projects differ from language to language. If CF can take packaged/zipped files then our deployment process will be the same for every app, regardless of what language is being used.
It would also be nice to be able to store artifacts as a single, packaged file and not have to un-package them every time I want to do a push.
A solution might be to have CF have a step to unpack/unzip a file BEFORE running the any of these detect.sh scripts for each build. That way the first thing it does after a file is uploaded would be to check if it needs to be unpackaged/unzipped and if so then do it then carry on like normal.

Hi @pivotal-manuel-morales
The CLI would not know whether your zipped file is intended to be deployed as a zip file (because e.g. you are using CF as a web server (static buildpack) to host a zip file, or is actually an archive with your app.
I'm not keen on introducing some flag to push to tell it that it's a compressed file that should be unpacked before pushing: then you would still have a different deployment process for this type of app. Also, the flag would basically be an alias for unzip myapp.zip | cf push myapp - I think our development resouces could be spent more wisely.
Note that the CLI unzips archives (and rezips them again) regardless, to send a list of the files within with their checksums to CC's resource matching API to see if there are any (big) files in CC's cache that we don't need to resend. So there is no performance gain to specifying an app archive as opposed to an "exploded" app.
I don't know the details of how a pushed app is presented to the buildpacks, but your suggested solution sounds better as it would be up to the buildpack (which can better determine whether unpacking an archive makes sense for the language) - as the Java buildpack does:
As you say, their detect.sh script could decide to peek inside the archive to see if they could run it if unpacked, and unpack if need be.
Why not send a feature request to the buildpacks team?
Regards,
Dies Koper
CF CLI PM

Hi @pivotal-manuel-morales
The CLI would not know whether your zipped file is intended to be deployed as a zip file (because e.g. you are using CF as a web server (static buildpack) to host a zip file, or is actually an archive with your app.
I'm not keen on introducing some flag to push to tell it that it's a compressed file that should be unpacked before pushing: then you would still have a different deployment process for this type of app. Also, the flag would basically be an alias for unzip myapp.zip | cf push myapp - I think our development resouces could be spent more wisely.
Note that the CLI unzips archives (and rezips them again) regardless, to send a list of the files within with their checksums to CC's resource matching API to see if there are any (big) files in CC's cache that we don't need to resend. So there is no performance gain to specifying an app archive as opposed to an "exploded" app.
I don't know the details of how a pushed app is presented to the buildpacks, but your suggested solution sounds better as it would be up to the buildpack (which can better determine whether unpacking an archive makes sense for the language) - as the Java buildpack does:
As you say, their detect.sh script could decide to peek inside the archive to see if they could run it if unpacked, and unpack if need be.
Why not send a feature request to the buildpacks team?
Regards,
Dies Koper
CF CLI PM

@pivotal-manuel-morales Did you send a request to the buildpacks team? Care to post a link to the issue here?

Closing, I don't think there is anything we can do.


[Table] TableHeader missing padding from Carbon update

Describe the bug
When upgrading to Carbon styles were introduced that removed the padding from non sorting Table headers. We need to add back in to avoid layout issues.
To Reproduce
Steps to reproduce the behavior:

Go to https://carbon-addons-iot-react.com/?path=/story/watson-iot-table--stateful-example-with-expansion-and-column-resize
Open dev tools
examine table header that is not a sorted row
See the lack of padding on the th

Needed for TBI release in June

:tada: This issue has been resolved in version 2.76.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


undefined method System\Classes\VersionManager::listNewVersions()

i try to use builder at first time but when i create plugins show that error not found listNewVersions();
i hope it can solve quickly.
thanks

You should update your OctoberCMS because in last version, there is definitely method listNewVersions():
https://github.com/octobercms/october/blob/master/modules/system/classes/VersionManager.php#L107

i try to run php artisan october:update
and result.
Updating October...No new updates found
i dont know what happen with update process on octobercms. should i do change it manually?

Try now. Build 316 should be released.

Btw what branch should be used for new projects? Master or stable? Master seems to be little bit behind. Is it safe to use stable?

Both are OK fine to use, master is RC, stable is v1.0.0 and uses Laravel 5.1 (LTS) and will be available in April and it requires PHP 5.5.9.


Use ForAttributeWithMetadataName

ForAttributeWithMetadataName was added to target types with attached attributes. Should have a performance boost - (not familiar with source generators).

Codecov Report

Merging #327 (88bffea) into main (dce4b42) will increase coverage by 0.19%.
The diff coverage is 93.60%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #327      +/-   ##
==========================================
+ Coverage   91.71%   91.90%   +0.19%
==========================================
  Files         109      117       +8
  Lines        3379     3707     +328
  Branches      438      502      +64
==========================================
+ Hits         3099     3407     +308
- Misses        189      205      +16
- Partials       91       95       +4

Impacted Files
Coverage Δ

...appings/NewInstanceObjectFactoryPropertyMapping.cs
100.00% <ø> (ø)

...pperly/Descriptors/Mappings/NullDelegateMapping.cs
83.33% <ø> (ø)

...escriptors/Mappings/ObjectPropertyMethodMapping.cs
90.00% <ø> (ø)

...riptors/MappingBuilders/QueryableMappingBuilder.cs
68.42% <68.42%> (ø)

...Descriptors/Mappings/EnumFromStringParseMapping.cs
73.68% <73.68%> (ø)

...criptors/Mappings/PropertyMappings/PropertyPath.cs
89.06% <80.00%> (-3.60%)
:arrow_down:

src/Riok.Mapperly/Helpers/SymbolExtensions.cs
86.56% <81.81%> (+0.85%)
:arrow_up:

...iptors/MappingBuilders/DictionaryMappingBuilder.cs
79.51% <83.33%> (+0.50%)
:arrow_up:

src/Riok.Mapperly/Descriptors/WellKnownTypes.cs
87.23% <83.87%> (-8.92%)
:arrow_down:

...ers/NewInstanceObjectPropertyMappingBodyBuilder.cs
80.43% <91.66%> (+2.83%)
:arrow_up:

... and 31 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Seems like the pipeline can't find 'GeneratorAttributeSyntaxContext'. I successfully ran the .Net Framework 48 integration test so I'm not sure what's failing

Seems like the pipeline can't find GeneratorAttributeSyntaxContext. I successfully ran the .Net Framework 48 integration test so I'm not sure what's failing

This depends on the Roslyn version which is used to run the source generator. AFAIK this depends whether the source generator runs in Visual Sutdio (which uses the Roslyn version which is shipped with VS), via cli (which uses the Roslyn version bundled by de sdk in use), or via other tools. Mapperly already multi targets Roslyn 4.0, 4.4 and 4.5 and provides a constant ROSLYN4_4_OR_GREATER. However, I'm not sure if the added complexity is worth it here.

This depends on the Roslyn version which is used to run the source generator.

Makes sense forgot about that 😆

I'm not sure if the added complexity is worth it here, wdyt?

I'd hoped this was a quick small change so hadn't given this much thought (haven't benchmarked a SG before). [https://github.com/dotnet/runtime/issues/70754](This PR) claims that roslyn may do up to x40 times less work with this change. The PR did raise some concerns, although I'm not sure if they apply here.
I'm not familiar with source generators so I can't say how beneficial this change would be, I've updated this PR to use ROSLYN4_4_OR_GREATER, I'll leave the final decision to you. You could revisit this if performance becomes a problem.

The biggest performance improvement could be achieved by implementing a "real" incremental source generator, as the current implementation is not really incremental (tracked in #72). I started working in that quite some time ago, but didn't find the time to complete it.

My knowledge of roslyn and source generators are limited to partially rewriting codemaid and reading these articles. I'm not sure if its possible to do memoisation when passing Compilation in a pipeline. I wonder how the dotnet team solve this for their source generators.
You could move the distinct call from Execute to the pipeline, assuming it caches it would probably only yield a minor improvement. 🤷
var distincted = mapperClassDeclarations.Collect().Select((myObjects, _) => myObjects.Distinct().ToImmutableArray());
var compilationAndMappers = context.CompilationProvider.Combine(distincted);

Do you use Mapperly in a project where Mapperly becomes a performance bottleneck?

Only in a small personal project

For what it's wortht the System.Text.Json does the exact same thing. It looks incredibly difficult to remove compilation while remaining correct. Perhaps a full rewrite could use compilation in the pipeline, get the type symbols, largely building most of the final syntax in type symbol form and then using a custom converter. Good luck 😁
I did think that most of the logic could be removed from Execute and turned into a pipeline step. If the resulting CompilationUnitSyntax is the same execute wouldn't run for that class node. I'm not sure how much of an improvement this would be and there would still be the problem of SourceProductionContext. It looks like it's only used for reporting diagnostics so perhaps it could be removed from DescriptorBuilder and some some alternative way of passing diagnostics from the pipeline into execute could be used.

Thanks for the help, I'll rebase when the tests pass.


Switch to images with a black background

[ ] WV
[ ] Pagination
[ ] Points
[ ] Lines
[ ] Polygons

Closing in favor of #7.


Automatically apply buildozer commands from warnings

Hi,
I've heard that the internal version of Bazel-watcher (iBazel) knows to watch for log warnings in Bazel outputs which are buildozer commands and to automatically apply them.
The need for this (for me) is to be able to automatically apply strict java/scala deps warnings.
This means a developer can use code from transitive targets, the rule will warn against it with a buildozer command (saying which target needs to be added explicitly where) and ibazel will automatically apply it.
Is there an interest to extend ibazel to that area?

Huh, this is a very interesting idea. I don't think I've yet run into that in iBazel but it doesn't mean that it doesn't do it. Do you have any ideas on how this would work? Are there any rules_ that are emitting buildozer commands to execute in case of failure?
In the specific case of Gazelle it might be reasonable to just blaze run //:gazelle whenever there is an import error but I'm not sure how to actually detect that. WDYT?

rules_scala does this
What is Gazelle?
BTW, @cgrushko is the one who told me about this feature in internal ibazel so maybe he has more info

IIUC it's nothing more than ibazel looking at the output of Bazel and running some commands when it sees something it recognizes (i.e., a regex --> command map).

The internal tool parses error messages from the compilation job for well
known patterns and then runs the suggested fixer tool (if there is any).
For Go code indeed, you'd probably just run gazelle, but for Java code you
often need very specific fixes (add a dependency to this target or that).

Given this PR https://github.com/bazelbuild/bazel/pull/3677 which adds
printing the buildozer command on strict_java_deps warn/error.
Can ibazel parse the message and just run the buildozer command?
On Mon, Jul 31, 2017 at 9:24 AM Martin Probst<EMAIL_ADDRESS>wrote:

The internal tool parses error messages from the compilation job for well
known patterns and then runs the suggested fixer tool (if there is any).
For Go code indeed, you'd probably just run gazelle, but for Java code you
often need very specific fixes (add a dependency to this target or that).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel-watcher/issues/18#issuecomment-318979236,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF039p-uEDPep07g56944IAjCg0h2ks5sTXMGgaJpZM4OmGod
.

That seems like an extremely reasonable thing to do. If you wanted to implement it with a whitelist of regexps (to prevent running something scary) scanning the output from Bazel I think you could implement that without a huge amount of effort.
Why don't you take a whack at it? If you run into problems, feel free to comment on here.

Useful notes to future implementers:
Go will give you access to the commands stdout pipe with cmdReader, err := cmd.StdoutPipe(). Rather than making the command's pipe be stdout you can intercept it line by line and print it and search for matching patterns.
A more complete example can be found at  https://nathanleclaire.com/blog/2014/12/29/shelled-out-commands-in-golang/

I have a rough and rudimentary version of this working. I'm working this in conjunction with unused_deps for rules_scala. I'm going to keep working on it and polish it up as we get unused_deps working. When I'm confident it works well I'll open a PR for the issue here.

Quick update on this: we have this working internally and are currently rolling ibazel + this change out to all of engineering. We plan to create a PR soon (1-2 weeks). If this is blocking anyone please let me know and I can try to get the PR ready faster than that.

At the moment, it is hardcoded to buildozer. The goal before merging this is to have there be a config file of some sort that provides regexes with capturing groups indicating the command and arguments.
If we wanted to extend it to do more than that, we could add a blacklist regex.
This should cover #2. As for #1, the blacklist should probably cover that. I'm not sure how it works with maven_jar. We have a fork of bazel-deps, which uses java_import_external and scala_import_external, so we can use strict/unused deps with maven dependencies. We're going to open a PR for that soon as well. It's made dealing with third party dependencies quite nice.

True. We're on a branch which has a jars_to_labels provider, so things are working for scala_import_external internally with buildozer suggestions. We'll work on getting PRs for all these open over the next couple weeks.
Sorry to drag this thread off topic :smile: