Support macOS architecture

Currently only iOS architecture is supported.
Add support to run Pact tests for macOS platform using Carthage as dependency manager.
 This will avoid confusion when importing/using the frameworks on the framework user (eg: linking to /Carthage/build/iOS/, and/or /Carthage/build/mac/ locations, then seeing two frameworks with same name in Xcode project navigator).

macOS target added and unit tests within the framework seem to work.
Commit hash: e86688ca0b77623174caca8e0608df7845643382
Hitting issues setting up a demo project for macOS architecture, PactSwift added as a dependency using Carthage.

The issue was with Sandboxing. Given a macOS app is not sandboxed, any path on the disk is fair game. Which is not ideal.
Solution is to write into NSHomeDirectory()/Documents only for macOS.
iOS/tvOS does not present with same issue.


Which version of the standard is implemented in the library?

Hello,
I tried to use your library for a little proyect but seems that the standard that I found at google does not represent the same as the one implemented here, I found the version 3.0 from 2012, is there a newer version? My question is because of the fact that the IMEI is specified at the standard to be a sequences of 15 bytes but in ASCII encoding, but if I print the message as written to the cursor I see that the IMEI is represented as bytes (starting at position 9 as array-like indexed)
1, 0, 34, 1, 0, 31, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 89, 208, 48, 0, 2, 0, 0 when it should be
1, 0, 34, 1, 0, 31, 0, 0, 0, 1, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 0, 0, 1, 0, 0, 89, 208, 48, 0, 2, 0, 0
Thank you!

Code was developed according to https://github.com/gadomski/sbd-rs/blob/e82295348474f3fb9751a301125a3790b3cc5bad/doc/Iridium Short Burst Data Service Developers Guide v3_0.pdf. The header is indeed stored as bytes, but there are expectations in the code that those bytes are valid utf8: https://github.com/gadomski/sbd-rs/blob/e82295348474f3fb9751a301125a3790b3cc5bad/src/mo/message.rs#L159-L176
Do you have some example data and/or code that show the problem?

My bad, I understand now what I was doing wrong. Thank you!

Sorry to reopen the issue, the length of MO Location Information should be 20 as shown in section 6.2.6 of the standard, am I right? I think the implementation has a 7 hard coded at https://github.com/gadomski/sbd-rs/blob/master/src/mo/information_element.rs#L138-L141

No problem! Agreed, that hardcode is incorrect. I think I grabbed looked down instead of right in this table:

I'll fix. Thanks for the catch.


Preview images are missing

Is the screenshot functionality broken? I've noticed the missing images a few weeks ago.

Same here.

Thanks. Just wanted to tell you how great this website is. It became my main way to browse HN nowadays.


:bug: Address issue with lack of keys

ℹ What is the context for these changes?

While working with the Tokengate component in a Hydrogen setting, I noticed that there was a violation of lack of unique keys in UnlockingTokens. When I went to resolve the issue I noticed that we were mapping through collections two times before rendering in a TokenList. This changes that implementation to map a single time and return an array of elements instead of an array of objects which are then mapped over to return an array of elements.
🕹 Demonstration

Errors from before:

I'm no longer getting the error when testing the component in Hydrogen.

🎩 How can this be tophatted?

✅ Checklist

[ ] ~Tested on mobile~ N/A
[x] Tested on multiple browsers
[ ] ~Tested for accessibility~ N/A
[ ] ~Includes unit tests~ N/A
[ ] ~Updated relevant documentation for the changes (if necessary)~ N/A

Okay, perfect, thanks for clarifying, I was just double checking! 👍


Update prefs-editor to 1.2.3

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

See https://github.com/caskroom/homebrew-cask/pull/31988 to know why this was closed.


v0.3.0 release candidate

Start a TL;DR chapter. #3
Move FAQ to dedicated chapter
Clarify contribution process to the book
Add version information in the book

@asdofindia r?

FTR, received a remote r+ from @asdofindia.


No Longer Seeking Committee Membership

I am withdrawing my request to join the community committee. Can I also have my name removed from the observer list? I will use the YouTube public stream to access future meetings.

Sorry to hear that you're not interested in joining anymore, Toby 😞
As linked above, I've submitted a PR to the tool we use to remove you from the observer list.

You've been removed. Thanks for being an observer, @tobyfarley ❤


[fedora] fedora 36 released

Fedora 36 is released.
release date: https://fedorapeople.org/groups/schedule/f-36/f-36-key-tasks.html
eol date: https://fedorapeople.org/groups/schedule/f-38/f-38-key-tasks.html

@istiak101 thank you :+1: :100: :1st_place_medal: :tada:
LGTM


Doing Quick Maths in Sol Gives Result Without Allowing Us To Copy Result

I used to use Raycast and it allowed you to do maths on the bar also.
The difference though is Raycast, when you type the enter bar, it will save the math result to the clipboard so you can paste it somewhere else.
Could you do this also in sol?
Thanks.

Pressing the enter key should copy the result to the clipboard. I'll take a look later, thanks for noticing!

Should be fixed in the latest version.


Support for addresses with staking keys #103.

Add stake keys to BpiWallet.
Generate stake keys in addWallets
withStakeKeys to add stake keys to TestWallets
Refactor key paths.
Set pcOwnStakePubKeyHash for BPI.
Pass WalletInfo's to contracts instead of PubKeyHash's.
Update tests with onEnterpriseAddresses helper.
TODO: Add tests.
TODO: Add another interface with lookups.
TODO: Add superduper interface with polyvariadic withContract.

I haven't added any tests yet. All the tests are left unchanged, that is they are initialized with enterprise addresses (no stake). Some of them rely on running with enterprise address.

TODO: Update changelog, but once we're settled with interface change

I reverted this pr to the last green commit and pr'ed latest changes instead to #136.
This is because my intention turned out to be impossible to implement ;/ more in that pr. We need to go with aproaches like the one at current revision.

Subsumed by #136


Add OBS-TQ

Allow the user to translate TranslationQuestions for OBS https://git.door43.org/unfoldingWord/en_obs-tq

Looks good in translationCore Create - v1.1.0-rc.6 build 21-ad9123a. Except for the font sizes not seeing any issues.

Looks good in v1.1.0-rc.7 build 22-ebe4a7c


Fixed bank-account tests

Update the Tests according to the How-to. If i miss something then please let me know.

LGTM. Are you happy with the concurrent testing @siebenschlaefer ?


Add to Let's Encrypt's Client Implementations List

Certes should be listed on https://letsencrypt.org/docs/client-options/
(Unfortunately for me, I did not find Certes for a while because it wasn't there, so I used a vastly inferior client for my application until I discovered Certes on Nuget)
Instructions and rules are here
If allowed, I can create the necessary pull request there.

That'd be awesome, feel free to to Certes to the list. 😄

Created PR https://github.com/letsencrypt/website/pull/320


Add pylint checking to GitHub workflow

See https://github.com/Senzing/stream-loader/issues/303

Pull requests:

https://github.com/Senzing/g2export/pull/36


after adding license metadata, update table -> Model list

model_id -> TheBloke/OpenAssistant-Llama2-13B-Orca-v2-8K-3166-GPTQ

Thank you so much Wauplin :). It is addictive :p Do you think I can try to complete the list or leave some for other people? :D

Thank you so much Wauplin :). It is addictive :p Do you think I can try to complete the list or leave some for other people? :D

Ahah feel free to complete as many as you can! :smile: And you can open a PR on Github for multiple PRs on Huggingface at once. Hacktoberfest is nearly over! :fire:

All right :) I will do some more. I was opening one for each so that its easier to check if everything I did is fine :)


Tibialyzer leaks process handles during memory scanning

It is my understanding that Tibialyzer has an internal timer that periodically scans Tibia's memory using OpenProcess/ReadProcessMemory. Assumedly, the code was taken from CodeProject, which usually isn't very good for serious programming due to severely lacking quality control. In any case, the process handle returned by OpenProcess should always be closed with CloseHandle after the scanning is done to avoid leaking open process handles and depleting system memory.

Should be fixed in commit 3d5a492b04dd76ed6fac862329a71d6bb93bc4cf.


Derive nonisolated members for protocol conformances

When deriving witnesses for protocol conformances within an
actor-isolated type, make those members 'nonisolated'. In the case
where this would work, for example because some of the state is
mutable, don't allow derivation of those witnesses.
Fixes rdar://90233250.

@swift-ci please smoke test


[channels,printer] fix backend load function

changed the signature of the backend loader function. the previous version casted the allocated pointer to UINT which might not be able to hold a pointer.

@freerdp-bot test properly please


⚠ TROJAN 🇺🇸 United States USO 1 has degraded performance

In 4beb047, TROJAN 🇺🇸 United States USO 1 (https://ust-2.optnl.com/) experienced degraded performance:

HTTP code: 200
Response time: 16166 ms

Resolved: TROJAN 🇺🇸 United States USO 1 performance has improved in 4d89039.


Ordering by NULLS FIRST / LAST

Hallo and thanks for this neat gem!
I'm trying to implement an order with some custom sql but I do not get results. I have a field posted_at that sometimes may be NULL. The order I want to achieve is posted_at DESC NULLS LAST
I tried using order_query in different ways but to no avail. Can you please tell me if it's possible and point me in the right direction?

What database are you using?

As a quick hack for now while we figure out a proper solution, you could do this:
order_query :my_order,
   [:posted_at_or_zero, :desc, sql: "COALESCE(posted_at, '0000-00-00 00:00:00')"]

def posted_at_or_date_in_the_past
  posted_at || DateTime.new(0)
end

Looks good! That's postgresql so I had to change order_query to COALESCE(posted_at, '1900-01-01 00:00:00') and posted_at_or_zero to posted_at || Time.utc(1900) since postgresql does not allow zeros as date.
That's good enough for my application. Nevertheless it would be nice if you could do something like: order_query :my_order, ['posted_at %order% NULLS LAST', :desc] and order_query would replace %order% by DESC or ASC. This way all future problems, where the name of a field to order by is not enough, would be solved.
Thank you for your time and your hints how to solve this problem!

I see that you did some work on this but it's well over a year old. I'm just curious what the status is and what your plans are for this feature.

The feature seems to work, but:

Tests are needed.
Branch needs rebasing.
Documentation needs updating.

I don't plan to do that myself, but if someone does all of the above and sends a PR I'll review it.

Thanks for the reply. I looked at it and rebased from master with no issues. All the tests ran afterwards but the nulls feature still needs a some work and is not currently working.
Nulls can be especially tricky since many times you want to keep them in the result set but still have them sorted.
As I looked more into it I realized I would be better off implementing keyset queries in my own app. I think this will be the case for most people who need queries that this gem doesn't handle.

Status update: Added more tests to the branch and got some of them to pass.

Status update: The nulls-order branch now passes all the tests. Still needs testing with all the databases to be safe to merge (#15)

All done! Released in v0.4.0!


[gym] Code signing of frameworks

I got error signing frameworks, while executing gym in ad-hoc export mode via Jenkins job.
Using terminal and running fastlane works, archiving via xCode also works, but executing as job via Jenkins web client fails with:
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/GUI.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/UIReusable.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/DataTypes.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/SportFeedsIO.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/AppStore.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/UIDataSources.framework
CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/Main-beicwtijpcchesevpcwexmyzfewm/Build/Intermediates/ArchiveIntermediates/App/InstallationBuildProductsLocation/Applications/App.com.app/Frameworks/Store.framework
(7 failures)

All those 7 frameworks are submodules in workspace, this failed at Embeded Frameworks with "Code Sign on copy" set.
I post question here, because I'm out of options. I've clear all provisionign prof., deleted account and downloaded all cert / prov. again. Clear all data that I know is connected with building app and xCode.
If someone got idea what I can do, I would be grateful (in beer kegs)
GYM lane:
		gym(
			workspace: workspace,
			scheme: scheme,
			configuration: config, //Release
			clean: true,
			include_bitcode: false,
			export_method: export, //ad-hoc
			# use_legacy_build_api: true, #when sth goes wrong, use this
			include_bitcode: false,
			# codesigning_identity: code_sign, #code sign not needed
			output_name: "#{scheme}.ipa")

Jenskins ver:  2.34
Fastlane ver: 1.111.0
Mac Pro (late 2013)
Mac Os Sierra Server
Xcode 8.1
Environment
| Gem                 | Version | Update-Status       |
| ------------------- | ------- | ------------------- |
| credentials_manager | 0.16.2  | ✅ Up-To-Date        |
| spaceship           | 0.38.4  | ✅ Up-To-Date        |
| fastlane_core       | 0.57.2  | ✅ Up-To-Date        |
| deliver             | 1.16.0  | ✅ Up-To-Date        |
| snapshot            | 1.16.4  | ✅ Up-To-Date        |
| frameit             | 3.0.0   | ✅ Up-To-Date        |
| pem                 | 1.4.0   | ✅ Up-To-Date        |
| cert                | 1.4.4   | ✅ Up-To-Date        |
| sigh                | 1.12.0  | ✅ Up-To-Date        |
| produce             | 1.3.2   | ✅ Up-To-Date        |
| gym                 | 1.12.1  | ✅ Up-To-Date        |
| pilot               | 1.12.1  | ✅ Up-To-Date        |
| scan                | 0.14.2  | ✅ Up-To-Date        |
| supply              | 0.8.0   | ✅ Up-To-Date        |
| match               | 0.11.0  | ✅ Up-To-Date        |
| screengrab          | 0.5.2   | 🚫 Update availaible |
| fastlane            | 1.111.0 | ✅ Up-To-Date        |

what is the output of gym? - can you provide fastlane env  - and share the Fastfile
are you using match to manage certificates/prov.prof?

I do not use match, we maintain certificates on our own.
<details><summary>🚫 fastlane environment 🚫</summary>

### Stack

| Key                 | Value                                       |
| ------------------- | ------------------------------------------- |
| OS                  | 10.12.1                                     |
| Ruby                | 2.3.1                                       |
| Bundler?            | false                                       |
| Git                 | git version 2.9.3 (Apple Git-75)            |
| Installation Source | ~/.rvm/gems/ruby-2.3.1/bin/fastlane         |
| Host                | Mac OS X 10.12.1 (16B2555)                  |
| Ruby Lib Dir        | ~/.rvm/rubies/ruby-2.3.1/lib                |
| OpenSSL Version     | OpenSSL 1.0.2g  1 Mar 2016                  |
| Is contained        | false                                       |
| Xcode Path          | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version       | 8.1                                         |

### System Locale

| Variable | Value       |   |
| -------- | ----------- | - |
| LANG     | en_US.UTF-8 | ✅ |
| LC_ALL   | en_US.UTF-8 | ✅ |
| LANGUAGE | en_US.UTF-8 | ✅ |

### fastlane files:

<details><summary>`./fastlane/Fastfile`</summary>

```ruby
#
# Project spec. Necessary things go here
#

	default_device = "iPhone 6s"

	devices_tests_ab = [
		"iPad Pro (12.9 inch)",
		"iPad Pro (9.7 inch)",
		"iPad Air",
		"iPhone 6s Plus",
		"iPhone 6s",
		"iPhone 5s"
	]

	schemes_tests_ab = [
	]

	schemes_tests = [
		"DataTypes",
		"GUI",
		"SportFeedsIO",
		"Store",
		"AppStore",
		"UIReusable",
		"UIDataSources",
		"App"
	]

	project_path_name = "App/App.xcodeproj"
	workspace_name = "Main.xcworkspace"
	scheme_release = "App"
	crashlytics_local_path = "App/Vendors/Crashlytics.framework/"
	sonar_name = "iOS-App"
	default_config = "Release"

	#Signing details, please fill it always!

#
# the end of needed things
#

platform :ios do

	desc "Upload on iTunes"
	lane :deploy do
		name = scheme_release
		provisioning_profile = provisioning_release

		build(export: "app-store")

		deliver(ipa: "#{name}.ipa",
				submit_for_review: false,
				skip_metadata: true,
				automatic_release: false,
				force: true)
	end

	desc "Run unit tests"
	lane :test do
		schemes = schemes_tests
		schemes.each do |scheme|
			test_generic(
				scheme:scheme,
				devices: [default_device])
		end
	end

	desc "Install all dependencies"
	lane :make do
		carthage(
			command: "bootstrap",
			platform: "iOS",
			use_binaries: true)
	end

	desc "Run build for fabric"
	lane :fabric do

		build_number = number_of_commits
		project_path = project_path_name
		workspace = workspace_name
		scheme = scheme_release
		crashlytics_path = crashlytics_local_path
		provisioning_profile = provisioning_adhoc

		version_number = get_version_number(
			xcodeproj: project_path)
		git_tag = "CI/v#{version_number}/##{build_number}"

		m_build_number = "Build number: #{build_number}"
		m_build_branch = "Branch: #{git_branch}"
		m_build_commit = "Commit: #{last_git_commit[:abbreviated_commit_hash]}"
		m_groups = ENV["CRASHLYTICS_GROUPS"]
		message = [
			m_build_number,
			m_build_branch,
			m_build_commit,
			"GIT TAG: #{git_tag}",
			"Groups: #{m_groups}"
		]
		.join("\n")

		increment_build_number(
			build_number: build_number,
			xcodeproj: project_path)

		build(export: "ad-hoc")

		crashlytics(
			crashlytics_path: "#{crashlytics_path}",
			api_token: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",

			build_secret: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
			notes: "#{message}",
			groups: m_groups)

		if !git_tag_exists(tag: git_tag)
			add_git_tag(
				tag: git_tag,
				)
			push_git_tags
		end

		slack(
			slack_url: "https://hooks.slack.com/services/xxxxx",
			channel: "xxxxx",
			message: message)
	end

	desc "Run metrics for all project modules"
	lane :metrics do
		schemes = schemes_tests
		schemes.each do |scheme|
			metrics_generic(
				scheme: scheme,
				devices: default_device)
		end
	end

	desc "Run Release Tests"
	private_lane :test_release do
		test
		test_ab
	end

	desc "Run all AB Tests"
	private_lane :test_ab do
		devices = devices_tests_ab
		schemes = schemes_tests_ab

		schemes.each do |scheme|
			test_generic(
				scheme:scheme,
				devices:devices)
		end
	end

	desc "Build project"
	private_lane :build do |options|

		export = options[:export]
		workspace = workspace_name
		scheme = scheme_release
		config = default_config

		gym(
			workspace: workspace,
			scheme: scheme,
			configuration: config,
			clean: true,
			include_bitcode: false,
			export_method: export,
			# use_legacy_build_api: true, #when sth goes wrong, use this
			include_bitcode: false,
			# codesigning_identity: code_sign, #code sign not needed
			output_name: "#{scheme}.ipa")
	end

	desc "Run metrics for selected scheme in project"
	private_lane :metrics_generic do |options|

		scheme = options[:scheme]
		devices = options[:devices]

		project_path = project_path_name
		workspace = workspace_name
		crashlytics_path = crashlytics_local_path
		project_name = sonar_name

		test_generic(
			scheme:scheme,
			devices:devices)

		slather(
			workspace: "#{workspace}",
			scheme: "#{scheme}",
			proj: "#{scheme}/#{scheme}.xcodeproj",
			cobertura_xml: true,
			jenkins: true,
			build_directory: "./build/#{scheme}.DerivedData",
			output_directory: "./build/#{scheme}.reports/")
		sh("cd .. && lizard ./#{scheme}/ -l swift --xml > ./build/#{scheme}.reports/lizard-report.xml")

		swiftlint(output_file: "./build/#{scheme}.reports/swiftlint.txt", ignore_exit_status: true)

		sonar_parameters = [
			"sonar.projectKey=#{project_name}.#{scheme}",
			"sonar.projectName=#{project_name}.#{scheme}",
			"sonar.junit.reportsPath=build/#{scheme}.reports",
			"sonar.junit.include=*.junit",
			"sonar.sources=#{scheme}/#{scheme}",
			"sonar.tests=#{scheme}/#{scheme}Tests",
			"sonar.swift.lizard.report=build/#{scheme}.reports/lizard-report.xml",
			"sonar.swift.coverage.reportPattern=build/#{scheme}.reports/cobertura.xml",
			"sonar.swift.swiftlint.report=build/#{scheme}.reports/*swiftlint.txt",
			"sonar.swift.workspace=Main.xcworkspace",
			"sonar.swift.project=#{scheme}/#{scheme}.xcodeproj"
		]

		final_sonar_parameters = "-D" + sonar_parameters.join(" -D")

		sonar(
			project_key:"#{project_name}",
			project_name:"#{scheme}",
			sonar_runner_args: final_sonar_parameters)
	end

    desc "Runs unit tests for selected scheme"
	private_lane :test_generic do |options|

		scheme = options[:scheme]
		devices = options[:devices]

		scan(
			scheme: scheme,
			code_coverage: true,
			devices: devices,
			derived_data_path: "./build/#{scheme}.DerivedData",
			output_directory: "./build/#{scheme}.reports/",
			clean: true)
	end

	desc "Called on error"
	error do |lane, exception, options|

		slack(
			slack_url: "https://hooks.slack.com/services/xxxxxx",
			channel: "xxxxx",
			message: "Exception: #{exception}")
	end
end

No Appfile found
fastlane gems

Gem
Version
Update-Status

credentials_manager
0.16.2
✅ Up-To-Date

spaceship
0.38.4
🚫 Update availaible

fastlane_core
0.57.2
✅ Up-To-Date

deliver
1.16.0
✅ Up-To-Date

snapshot
1.16.4
✅ Up-To-Date

frameit
3.0.0
✅ Up-To-Date

pem
1.4.0
✅ Up-To-Date

cert
1.4.4
✅ Up-To-Date

sigh
1.12.0
🚫 Update availaible

produce
1.3.2
✅ Up-To-Date

gym
1.12.1
✅ Up-To-Date

pilot
1.12.1
✅ Up-To-Date

scan
0.14.2
✅ Up-To-Date

supply
0.8.0
✅ Up-To-Date

match
0.11.0
✅ Up-To-Date

screengrab
0.5.2
🚫 Update availaible

fastlane
1.111.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.0.0

executable-hooks
1.3.2

bundler-unload
1.0.2

rubygems-bundler
1.4.4

bundler
1.13.1

io-console
0.4.5

highline
1.7.8

commander
4.4.0

terminal-table
1.4.5

json
1.8.3

multipart-post
2.0.0

faraday
0.9.2

faraday_middleware
0.10.0

dotenv
2.1.1

net-ssh
3.2.0

net-sftp
2.1.2

plist
3.1.0

rubyzip
1.1.7

security
0.1.3

krausefx-shenzhen
0.14.11

slack-notifier
1.5.1

i18n
0.7.0

thread_safe
0.3.5

tzinfo
1.2.2

minitest
5.8.3

activesupport
<IP_ADDRESS>

colored
1.2

claide
1.0.0

xcodeproj
1.3.1

rouge
1.11.1

xcpretty
0.2.4

terminal-notifier
1.6.3

addressable
2.3.8

credentials_manager
0.16.2

multi_xml
0.5.5

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20160826

http-cookie
1.0.2

faraday-cookie_jar
0.0.6

fastimage
1.6.8

babosa
1.0.2

xcode-install
2.0.6

word_wrap
1.0.0

multi_json
1.12.1

excon
0.45.4

gh_inspector
1.0.2

fastlane_core
0.57.2

mini_magick
4.5.1

xcpretty-travis-formatter
0.0.4

uber
0.0.15

representable
2.3.0

retriable
2.1.0

mime-types-data
3.2016.0521

mime-types
3.1

hurley
0.2

little-plugger
1.1.4

logging
2.1.0

jwt
1.5.6

memoist
0.15.0

os
0.9.6

signet
0.7.3

googleauth
0.5.1

httpclient
<IP_ADDRESS>

google-api-client
0.9.14

generated on: 2016-12-05

```

Nevermind, my stupid is limitless - I've delate unlock_keychain from Fastfile because of error, but Ive reverted that change and add proper path: parameter. It is working now.
This can be closed.


[REQUEST] Make HTTP a folder and extract methods according to their endpoint

See title

Closing as the associated PR has been merged.


Install YOLOv8 Dependencies

Description:
Install required YOLOv8 libraries and dependencies (e.g., Ultralytics, PyTorch).
Definition of Done:

[ ] YOLOv8 dependencies are installed without errors.

Testing Notes:

[ ] Verify installation by running YOLOv8's default example.
[ ] Test installation on both local development and edge devices (Raspberry Pi).

In this issue you simply run the YOLO model and see if it opens the camera and starts detecting stuff


protoc-plugin assumes protobuf file extension is .proto

While trying to translate a protobuf file with another file extension as .proto protoc-gen-rust crashes with the following backtrace:
RUST_BACKTRACE=1 protoc --rust_out=. test.pb
thread 'main' panicked at 'explicit panic', protobuf-1.1.0/src/lib/strx.rs:10
stack backtrace:
   1:     0x558bf4fc473c - std::sys::imp::backtrace::tracing::imp::write::h9c41d2f69e5caabf
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
   2:     0x558bf4fc6b9e - std::panicking::default_hook::{{closure}}::hcc803c8663cda123
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/panicking.rs:351
   3:     0x558bf4fc67a4 - std::panicking::default_hook::hd5bda4e453dfb4be
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/panicking.rs:367
   4:     0x558bf4fc6f7b - std::panicking::rust_panic_with_hook::hffbc74969c7b5d87
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/panicking.rs:555
   5:     0x558bf4f6bf7f - std::panicking::begin_panic::hd72c3f5a7d2ab8d4
   6:     0x558bf4fb5faf - protobuf::descriptorx::proto_path_to_rust_mod::hafa8ea57a3b75135
   7:     0x558bf4fb1544 - protobuf::codegen::gen::h77101cd7554ccfae
   8:     0x558bf4fb2cec - protobuf::compiler_plugin::plugin_main::hee20f29dfc9170fb
   9:     0x558bf4fcf57a - __rust_maybe_catch_panic
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libpanic_unwind/lib.rs:98
  10:     0x558bf4fc76e6 - std::rt::lang_start::ha0568cc91d8c5b09
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/panicking.rs:436
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/panic.rs:361
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/rt.rs:57
  11:     0x7f9e1ede03f0 - __libc_start_main
  12:     0x558bf4f6a399 - _start
  13:                0x0 - <unknown>
--rust_out: protoc-gen-rust: Plugin failed with status code 101.

Translating the same file using protoc --cpp_out . test.pb works fine, renaming it to .proto and translate it afterwards also works fine. At least the panic message should contain an information about this.

I've added better panic message.
If you think rust-protobuf should handle files with different suffixes than .proto, please reopen the issue.


week view headers misaligned with day columns

Do you want to request a feature or report a bug?
Bug report
What's the current behavior?
on:
 "react-big-calendar": "^0.20.2"
  "react": "^16.5.2",
 "react-ace": "^5.8.0",

Week view (and day view on narrow screens) has completely misaligned headers with the day columns

What's the expected behavior?
The headers are aligned like  in this sandbox
I looked at the markup differences between the sandbox and my code and here is an image showing it:
 with the main differences highlighted.
Aside from the rbc-overflowing class being added the to the top level header component there are a number of markup differences in the structure as well.
The code for init is:
<Paper style={{marginTop: 10, height: '85vh'}}>
    <BigCalendar
        drilldownView="agenda"
        // onEventDrop={this.moveEvent}
        events={events || api[showing].data()}
        onSelectEvent={ (event) => api[showing].navigator(push)(event)}
        defaultView='month'
        scrollToTime={moment().toDate()}
        eventPropGetter={(event,start,end,isSelected)=>{
            return {
                style: {
                    backgroundColor:'transparent',
                    color: Colors.primary,
                    borderRadius:0,
                    border: 'none',
                    whiteSpace: 'pre-wrap',
                    overflowY: 'auto'
                }
            }
        }}
        components={{
            event: this.Event,
            agenda: {
                event: this.EventAgenda
            },
            toolbar: CustomToolbar,
        }}
    />
</Paper>

I have this same bug and haven't found an answer anywhere. Did you end up finding a solution?

I removed a surrounding div and got this to work so closing - be careful what you surround this in!!

I managed to run into this by creating a RBC demo in CRA. The reproduction is as follows:

Create a new CRA app
Remove all CRA's styles
Replace App.tsx with the following code:

import React from 'react';
import 'react-big-calendar/lib/css/react-big-calendar.css';
import BigCalendar from 'react-big-calendar';
import moment from 'moment';

type AppProps = {};

type AppState = {};

type CalendarEvent = {
  start: Date;
  end: Date;
  title: string;
}

export default class App extends React.Component<AppProps, AppState> {
  private static readonly localizer = BigCalendar.momentLocalizer(moment);

  render() {
    const start = new Date()
    start.setHours(10);
    const end = new Date();
    end.setHours(22);
    return (
      <BigCalendar<CalendarEvent>
        localizer={App.localizer}
        events={[ { start, end, title: 'An event' } ]}
        defaultView="week"
      />
    );
  }
}

Upon loading the page, the day column headers will be completely off screen. Switching the view to month and back fixes that. This might be a different issue that manifests the same way.
Please reopen this @buddythumbs as in my case this issue doesn't seem to be fixable by removing the surrounding container (I do need to keep at least one div as React won't allow me to render to body directly) and it reproduces consistently.

I'm still having the issue. As for @TomasHubelbauer I can't remove the surrounding block and because of this I can't update my header. Any fix?

Hi,
for anyone having this issue, not sure if its considered a fix. but based on the rouge css hint by someone earlier, I've commented flex-basis:auto !important on class .rbc-header on _calendar.scss. hope this helps someone. now my header is aligned with day.


runtime: arm FreeBSD should work around missing ELF auxiliary vectors

What version of Go are you using (go version)?
Go 1.16.5 as provided by the FreeBSD ports tree in lang/go.
Does this issue reproduce with the latest release?
Not tested.
What operating system and processor architecture are you using (go env)?
arm with GOARM=7 on arm64 FreeBSD 13.0-RELEASE.
What did you do?
Attempt to build and run any armv7 Go program in an armv7 jail on arm64 FreeBSD 13.0-RELEASE.
What did you expect to see?
A working Go program.
What did you see instead?
runtime: this CPU has no floating point hardware, so it cannot run
this GOARM=7 binary. Recompile using GOARM=5.

Additional notes
Due to a bug (PR #256897, PR #256903) in FreeBSD 13.0-RELEASE, the AT_HWCAP and AT_HWCAP2 ELF auxiliary vectors are not provided to arm programs running under arm64 FreeBSD 13.0-RELEASE.  Unfortunately, the Go runtime uses these vectors to check for floating point support and erroneously believe that no such support is available if the auxiliary vectors are missing.
To make running arm Go programs on arm64 FreeBSD 13.0-RELEASE possible, I suggest to change the floating point support detection code in src/runtime/os_freebsd_arm.go to be changed such that it checks for the presence of AT_HWCAP and AT_HWCAP2 and only denies execution if both vectors are present and indicate lack of support.  If the vectors are not present, we are on a system that indicates neither presence nor absence of floating point support via the ELF auxilary vectors and proceed with execution.  I believe that only affects arm64 FreeBSD 13.0-RELEASE which should always have GOARM=7 compatible floating point support as that is architectural for ARMv8-A.

questions:

To reliably distinguish between not supported features and auxiliary vector not initialised: If the arm auxiliary vector is populated does it have at least one bit set on any arm platform that FreeBSD supports?
I dont think Go runtime knows if its running a arm binary on arm64 host or arm binary on arm host. So it can not currently use that information to assume all of ARMv8-A is supported. If Go should detect running on an arm64 host that how would it do that on Freebsd with low overhead and early in the runtime (therefore ideally from the aux vector itself)?
Is the request to assume GOARM=x is supported when auxiliary vector is known to be not initialised and binary is compiled with GOARM=x and/or that all of GOARM=7 is supported when host is detectable as being ARMv8-a?
Should only the runtime checks be skipped or should the internal/cpu feature bits actually be set to assume the features are supported.
Is it correct to assume FreeBSD will fix the bug to correctly fill the auxiliary vectors in the future?

We may look into a workaround (if easy) for go1.17/go1.18 but im not sure since this is a bug in FreeBSD and has a workaround (compile with GOARM=5) this qualifies for a backport. I would assume the number of active uses of running go
arm binary on a arm64 host are low.

Looking at the PR it seems the HWCap auxilary vectors are not provided at all (not just 0). Therefore the easiest workaround seems to record a bool in runtime for FreeBSD and if its not true skip the runtime checks for goarm. Likely with a hint that the expected aux values were not supplied.

Hi @martinsch, let me try to address your questions:

On the defective version of FreeBSD, this means the auxiliary vector is not present, not just not populated.
One option would be to check the kern.supported_archs sysctl for presence of the string aarch64 indicating support for running aarch64 binaries (implying an ARMv8-A CPU), but this might be complex to implement.   As for another option, the hw.machine_arch sysctl could be used.  This sysctl gives armv6 on ARMv6 platforms and armv7 on ARMv7 platforms (older version of FreeBSD may have also returned armv5).  This sysctl has a fixed number (see HW_MACHINE_ARCH in sys/sysctl.h) and can thus be retrieved easily.
The request is to assume the former with the intent that the runtime is not able to detect if the binary can run successfully, so it'll give it a best effort.  The only system where this is known to happen is based on ARMv8 on which this works out, so it seems to be a reasonable compromise between ease of implementation and usefulness.  Perhaps a floating point instruction could be placed into the runtime initialisation code path in this case so the binary at least crashes as early as possible if it turns out the CPU does not support the required instructions.
I don't know.
I am certain this will be fixed for 13.1-RELEASE as it seems to be a rather important issue.  It might even be fixed in an upcoming patch release to 13.0-RELEASE but I can't say.

The main purpose of running armv7 Go on arm64 is to build the FreeBSD ports tree (I believe this is why this support was added in the first place).  Ever since the advent of fast arm64 machines, this has been desirable over building armv7 ports on QEMU and AFAIK the armv7 ports builders run by the FreeBSD project do it this way.
So this issue means that no port based on Go has armv7 binary packages available at the moment.  It also means that Go developers on FreeBSD cannot test their software on armv7 on a machine with arm64 kernel and will have to buy a separate ARM machine for armv7 testing.

Thanks for the context. I think we can just reintroduce the code removed in: https://github.com/golang/go/commit/3b6216ed0601c81fe42c2a4738d419afccb62163# with a new comment explaining FreeBSD 13 Release doesnt set HWCap for arm on arm64 hosts.
I will send a CL early next week and let the go release team decide if they allow a backport or if FreeBSD team should patch it in until its resolved in FreeBSD.

@martisch Any progress with this one?

@dmitshur @golang/release would it be possible to add the reintroduced workaround for missing AT_HWCAP into go 1.17 release (or if preferred a patch release)?
@clausecker please test if https://golang.org/cl/335791 works in an armv7 jail on arm64.

Any progress on the CL?
Meanwhile, FreeBSD has fixed the issue kernel side and the fix is slated to appear in FreeBSD 13.1-RELEASE.  However, that release is not going to be released until next year I believe.

The CL would need to be accepted as a backport. This I can not ratify alone. Without a backport this may land in go1.18 if accepted, but it seems this would by the time be solved by FreeBSD 13.1-RELEASE itself. So it seems the only question is if this warrants a back port or not to work around omissions of setting HWCAP in the FreeBSD 13 Release.
/cc @aclements


Implementations must support IMarshal

Implementations (e.g. WinRT delegates and those created with the implement macro) support IAgileObject but IMarshal is also required for some languages.

No action needed.
Just scribbling a note here that the Real Time Stylus object falls in this bucket. Failure to implement this will result in a E_INVALIDARG return from IRealTimeStylus::AddStylusSyncPlugin.


JfxHighlighter not working properly in ListView

Hi,
I tried to use the JfxHighlighter to highlight the contents in the listview. I used the below line
JFXNodeUtils.addDelayedEventHandler(searchText, Duration.millis(400), KeyEvent.KEY_TYPED, event -> jfxHighlighter.highlight(availableSignalsList, searchText.getText()));
It works properly until you scroll on the list view. My filtered list is so big that i need to scroll the listview. When i scroll the listview,
it is not highlighting anything.

Hi,
ListView updates the cells contents on scroll, thus you need to add some scroll listeners to refresh the highlighter.
Regards,


when would your code be released?

Excellent job! Waiting for your codes!!

Thanks @1999kevin! We'll have the code up soon, just performing some clean-up

It is now @1999kevin, sorry for the delay!


Move Openverse API and catalog to openverse.org subdomains

Summary
Move the infrastructure which currently exists on openverse.engineering to openverse.org.
Description

Pseudo-project proposal (collapsed to preference the issue and document list)
We currently pay for two Cloudflare accounts, one for openverse.engineering and openverse.org. If we move the API and catalog to live on subdomains of openverse.org, we should be able to change our openverse.engineering account to a free one, saving 200 USD a month that can go towards e.g., Plausible.
In the initial discussion we had about this we only talked about moving the API. However, because Airflow is behind Cloudflare Access, we may need to move it as well. I've tried to understand whether that is the case based on the Cloudflare pricing and it seems like Cloudflare Access might be free under our current usage but it isn't clear to me whether that's free for specific paid account types or free for any account type.
The end result of this project should be that openverse.engineering and its usage should be entirely covered by a free Cloudflare account. The planning for this project must consider the various features we will need that will cause contingencies here:

Do we need to route redirects through Cloudflare, or could we do it AWS side with a small redirect service? If we do it via Cloudflare, we might need individual page rules for each domain we are redirecting. For the API there are at least two we need to redirect: api.openverse.engineering and api-production.openverse.engineering. Do we need to redirect staging (api-staging)? What about the legacy staging subdomain, api-dev? We also continue to redirect search.openverse.engineering, our legacy frontend domain(s). Cloudflare free only supports three page rules. We currently have 3 redirects already (for the frontend). We would need at minimum one more for the API (api.openverse.engineering) but to match our existing redirect philosophy with the frontend, we should redirect api, api-production and api-staging. That would lead to 6 minimum page rules if we used page rules for the redirects. The other existing page rules are caching related and would be moved into openverse.org, so they do not need to count towards our total page rule utilisation.
Can the catalog stay on openverse.engineering if it is a free account? Namely, as discussed above, do free Cloudflare accounts support Cloudflare Access? If we manage Cloudflare Access on two different Cloudflare zones, the access terraform module will need to be updated or potentially completely re-written to accommodate multiple Cloudflare zones.

Additional considerations:

The jumphost is on openverse.engineering. Should is stay there?
If we need to configure more redirects than page rules allow on a free account, can a small ECS Nginx or caddy instance be configured to properly handle every redirect we need? Is it cheaper to host an ECS service for this or a small EC2 instance or can it be managed entirely through AWS ALB listener rules instead?
Special attention must be paid to timing the transitions and announcements on Make. Should a notice be sent to registered API users?
Updates to API domains will be necessary in many parts of our stack, including documentation and the frontend configuration. Frontend test tapes will need to be updated for the new domain. Are there other instances of such fixtures that we would need to update?

A task list was written by @zackkrida and @dhruvkb before. It is listed below in a collapsed element as I think it should be referenced with a grain of salt. It demonstrates the overall picture well, especially for communications, but hides what I think are probably the most complex parts of this (namely the individual infrastructure steps we need to take for the first task).

The task list created during the original, internal discussion and proposal of this project.

Infrastructure PR to point the new domains (can be done in advance) and allow traffic from openverse.org to the load balancer API service
Add Cloudflare page cache rules for the API in openverse.org
Follow-up infrastructure PR to switch the old domains to 301 redirects
Write a make post
Get a list of all unique registered email addresses (pretty easy with the jumphost and a DB query) and draft an email
Publish the make post and send the email
Update references to https://api(-staging)?.openverse.engineering in the documentation
On launch day: merge the infra PR to turn the old domains into redirects
On launch day: Downgrade the Cloudflare account for openverse.engineering
On launch day: Announce completion of the redirect on the make post and in make slack

Documents

Because this project is relatively clear in its motivations and requirements, we will skip the project proposal. This project thread's description will serve as a general project proposal.

[x] https://github.com/WordPress/openverse/issues/2038

Issues

Issues are mostly in the infrastructure repository, organised into the following four milestones.

[x] Preliminary work
[ ] Airflow work (In progress)
[ ] API work (Available for work)
[ ] Finalisation work (Blocked)

Preliminary work blocks the API and Airflow specific work. API and Airflow work can happen in parallel. Finalisation work is blocked by everything else.
Additionally, these four issues are in the monorepo related to this:

https://github.com/WordPress/openverse/issues/3741
https://github.com/WordPress/openverse/issues/3742
https://github.com/WordPress/openverse/issues/3743
https://github.com/WordPress/openverse/issues/3821

Here are my answers to the questions I raised in the issue description:

Do we need to route redirects through Cloudflare, or could we do it AWS side with a small redirect service?

If there are not sufficient Cloudflare rules in the free tier to redirect everything that needs it, we can use AWS load balancer rules. I don't have a strong preference either way, maybe with a slight lean towards Cloudflare because the rules are more flexible, and technically they're at the edge, and would perform slightly better than load balancer rules. But those benefits are super negligible, so if there is any hassle at all making it work in Cloudflare, AWS LB rules are a perfectly good fallback that will 100% work and will not cost us anything.

Can the catalog stay on openverse.engineering if it is a free account?

No reason to do this. Let's move everything over and call it a day.

The jumphost is on openverse.engineering. Should is stay there?

This is no longer true. The new SSH bastion is on openverse.org

can a small ECS Nginx or caddy instance be configured to properly handle every redirect we need?

NO. We will use AWS load balancer rules. We must avoid any solutions that introduce new services!

Should a notice be sent to registered API users?

Yes.

Updates to API domains will be necessary in many parts of our stack, including documentation and the frontend configuration. Frontend test tapes will need to be updated for the new domain. Are there other instances of such fixtures that we would need to update?

The implementation plan will cover this.

Implementation plan is approved and merged (thanks @dhruvkb and @AetherUnbound for the careful review).
I've created all the issues for this project. They are primarily in the private infrastructure repository, almost all the work will happen there. The milestones are as follows:

Preliminary work
Airflow work
API work
Finalisation work

Preliminary work blocks the API and Airflow specific work. API and Airflow work can happen in parallel. Finalisation work is blocked by everything else.
Additionally, these three issues are in the monorepo related to this:

https://github.com/WordPress/openverse/issues/3741
https://github.com/WordPress/openverse/issues/3742
https://github.com/WordPress/openverse/issues/3743

Thanks Sara for all this careful planning! I've gone ahead and moved all of the non-blocked tickets in the first preliminary work milestone into our TODOs.

Thanks for mentioning the blocked issues in the preliminary work milestone. Those are actually finalisation work, I'd just forgotten to move them! They're in the correct milestone now :+1:

Preliminary work for this is finished now. I will start working on preparing the Airflow migration now.

I wanted to make a note here concerning CloudFlare and some of the currently-manual configuration for dealing with bots. On the frontend we now have "super bot fight mode" enabled, which automatically blocks all traffic from known and likely bad bots, while allowing "verified" web crawlers like Internet Archive, Google, Bing, etc. to access the frontend.
After moving to the openverse.org domain, we probably want to create Web Access Firewall rules to skip "super bot mode" rules for the API. I would guess at least that our API users programmatically accessing the API would be marked as bots by Cloudflare and blocked by these rules.
Specifically, our WAF rules need to skip the "http_request_sbfm" (sbfm = super bot fight mode) request phase. I think the whole rule would (roughly) look like this:
# https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/ruleset
resource "cloudflare_ruleset" "skip_sbfm_for_api" {
  zone_id = var.cloudflare_zone_id
  name    = "Skip Super Bot Fight Mode for the API"
  kind    = "zone"
  phase   = "http_request_sbfm"

  rules {
    action     = "skip"
    expression = "(http.request.uri.path matches \"^/api\")"
    description = "Skip Super Bot Fight Mode for requests to the API"
  }
}

@zackkrida Can you please add this note to the issue for moving all existing Cloudflare rules from the .engineering zone to the .org zone? It's unlikely the implementer of that issue would refer to this project thread before starting it, so it's more reliable to share that information there. The issue should have all the information, ideally.
https://github.com/WordPress/openverse-infrastructure/issues/777

@sarayourfriend I've updated my comment to match the hostname correctly. I've also manually added this rule to Cloudflare now, so no action needs to be taken in https://github.com/WordPress/openverse-infrastructure/issues/777.

Is the main issue needing to make sure that the existing rule for the frontend in the .org zone not accidentally cause an issue to the API?

Yes, the goal is so that Super Bot Fight Mode doesn't block programmatic API traffic once it's moved over to openverse.org. We do not need to bypass this for Kibana or Airflow.

I've also manually added this rule to Cloudflare now

As in, added to the .org zone? Will you open a PR to add it to the cloudflare root module of the infrastructure repository? Ideally we move away from the practice of manually defining rules in Cloudflare without reviews or the chance to document in comments on them, particularly for things we expect to exist for a long time or indefinitely.

@sarayourfriend agreed on no longer manually defining these in the cf ui,I mentioned in my edit that we should move this firewall rule with the others.

Okay, thanks.

Yesterday I got up the PR to deploy Airflow with Ansible on a stable EC2 instance: https://github.com/WordPress/openverse-infrastructure/pull/829
Staci's early review comment made me realise I hadn't mentioned in the project thread that the security group refactor described in the implementation plan is not workable. Details of that are described in this comment on the issue that was meant to implement the abstracted security group module.
To summarise: the proposed abstraction in the implementation plan largely misses the point of security groups and how to best organise them. Rather than enforcing a uniform basic standard of ingress/egress rules by applying those rules to each security group individually, we should add instances to relevant security groups configured with the rules relevant to them. We can have several thousands of security groups, and with our service volume we will not run into the limit, even if we had an individual security group for each and every rule (which we don't need). Rather than configuring each EC2 instance's security group with SSH ingress rules, we should just add the instance to a shared security group with those rules.
That will be a long term refactor that I need to sit down and plan out into discrete tasks. We cannot do it as part of this project without causing significant delays to it as well as disruptions to virtually all other ongoing infrastructure work. I don't think this needs an implementation plan, it just needs someone (very likely me) to sit down and look at all our existing security groups to identify the places that need changes. Another issue that complicates this is the need to migrate away from both inline security group rules in Terraform and away from the old security group rule resources, all towards the new rule resources which have the significant benefit of leveraging AWS's relatively new security group rule ids, which neither of the previous approaches (which are the ones we use) implement.
So, we will still eventually change how we manage security groups to reduce duplication, we just won't do it the way the implementation plan suggests, and we will not do it as part of this project.

Airflow is live at airflow.openverse.org :tada:
A final infrastructure PR is up to finalise the Ansible, compose, monitoring, and IAM configuration. Please review this as soon as possible, @WordPress/openverse-catalog @WordPress/openverse-infrastructure. The resources are live in production and ideally these changes are merged to main soon.

The API is now available at api.openverse.org :tada:

I've drafted text for the Make post and email https://docs.google.com/document/d/1ESmzbH6vkp8rxJBsy3P0_BLZ01sgKVFAQa3LnPSvBhQ/edit?usp=sharing
@WordPress/openverse-maintainers, please review the text. I'll start working on the baseline requirements for the management command in https://github.com/WordPress/openverse/issues/3742.
I'll shortly have a PR up for #3741.

I've got the make post up and scheduled, with a switch-over date of 3 June 2024, which @zackkrida and I just decided on. That gives a month of lead time, with the post scheduled to publish 6 May at 00:00 UTC.
This PR introduces the management command for sending the email to registered and verified API users: https://github.com/WordPress/openverse/pull/4229
Finally, https://github.com/WordPress/openverse-infrastructure/pull/876 updates the canonical URL and introduces a staging-only redirect for testing.

The Make post went out earlier this week: https://make.wordpress.org/openverse/2024/05/06/the-openverse-api-is-moving-to-api-openverse-org/
I got all set up to run the management command to send notifications to registered API users, but found an issue with the query in the original code, and it only pulled 2 email addresses in production to send to. That's not the expected outcome. I dug deeper, and realised we made a mistake in how we wrote the query. We'd written the query off of the OAuth2Verification model, thinking that it would be an easy way to find the email addresses of verified registrations. However, we delete the verification row once the email is verified, so that's not a sound approach.
Instead, we need to query off the registration table where name is the name of a verified application. This is safe because name  is unique indexed on the registration table.
I've confirmed that with this re-written query, we get a more expected number of emails to send in production.

The emails announcing the API move are sent as of 2024-05-08T00:59:58.366Z.

The redirects are LIVE! Everything appears to be working. I tried out Gutenberg and Jetpack integrations and both look to be fine as far as I can tell.
I've removed https://github.com/WordPress/openverse-infrastructure/issues/781 and https://github.com/WordPress/openverse-infrastructure/issues/782 from the milestones for this project because we agreed they should not block the "shipped" status of this project, and I wanted to make that clear based on the milestone.
With that, the API tickets are done, and I've closed the milestone!
I'm going to start working on https://github.com/WordPress/openverse-infrastructure/issues/785 now, which will free up https://github.com/WordPress/openverse-infrastructure/issues/787 very soon :tada:.

@sarayourfriend sadly, the redirect broke the Inserter > Media > Openverse flow in Gutenberg and (surprisingly to me) the Gutenberg E2E tests, which rely on availability of the Openverse API. There was some chat about this in the Make WP Slack (public, but requires an account).
I made an infra PR to revert the change, and a Gutenberg PR was already merged to replace the URLs to use openverse.org.
To keep old versions of WordPress working, now, we would have to keep openverse.engineering operational indefinitely. That is clearly infeasible, so instead, I think we should re-implement the redirect right after WordPress 6.6 launches.
I also added a status update to our https://make.wordpress.org/openverse/2024/05/06/the-openverse-api-is-moving-to-api-openverse-org/ make post for the change, and pinned that post as we figure out next steps.

We've got a solution to redirect everything except the media inserter. The solution doesn't prevent the goals of this project from succeeding as it only uses free Cloudflare features and can theoretically exist indefinitely. We can discuss when specifically we would remove it, whether that's with the 6.6 launch or later, in the follow up to this work.

Once https://github.com/WordPress/openverse-infrastructure/pull/920 is merged this project can be moved from shipped to success :partying_face:

I've just applied and merged https://github.com/WordPress/openverse-infrastructure/pull/920!
This project is complete :tada:


Changing display language needs restart to take effect, but user is not notified of the same

Changing display language (either via the Configure Display Language command or editing the locale.json file directly) needs restart to take effect, but user is not notified of the same.
I would suggest that we show a prompt that would say something like A restart is required for the change in display language to take effect. Options being Restart and Dont show again.

Makes sense

The Configure Display Language command seems to just open locale.json for editing. locale.json has default contents:
{
	// Defines VSCode's display language.
	// See https://go.microsoft.com/fwlink/?LinkId=761051 for a list of supported languages.
	// Changing the value requires restarting VSCode.
	"locale":"en"
}

Do we need more notification than that line right above? If so should this be implemented on save? Or what?

Honestly, thats the first time I saw that message in the comments. Even though... its right there!
I believe having a notification still helps. Especially because in this case we need a restart (close all windows and start again. In Mac, quit and restart) and not just a reload on the single window.
It should show up after the file is saved if platform.locale is not the same as what was just configured. I am not sure if we are firing any events when the locale is changed. If we do, then you can piggy back on that.  If we don't and its not trivial to add one, then I am ok with closing this issue given that the comment has the instruction to restart.


Fix issue with mobile number normalization.

What's this PR do?
This fixes a bug with normalizing mobile numbers (accidentally trying to strictly compare a '1' with 1, and so always skipping the conditional). This means some accounts will still have been saved with leading 1s in the production database.
It also uses the Registrar's normalize methods for the User model's mutators to DRY things up. Well, there seems to be some sort of strange error when trying to resolve the Registrar from the User model... keeping things WET for the moment, and will extract to a "Normalizer" in a future PR. 💦
Fixes #409. References #357.
How should this be reviewed?
The failing test added in 382df67 should :sos:, and then it should be ✓ in 8853a65... okay but then it should really be ✅ in 10e7ec7. Great work.
Checklist

[ ] Documentation added for changed endpoints.
[x] Tests added for new features/bug fixes.

For review: @angaither @weerd
/cc @sergii-tkachenko

One quick comment, but looks good! 👍

💦

I also ended up undoing the fix in the registrar (2fe6020), since that would prevent Sergii from being able to look up existing non-normalized records for his current work. Will re-apply once all the production data is cleaned up.

👍

👍


Updated the definition of 'Reducer' and 'Dispatch' for TypeScript 2.3+

I am sorry for nagging, in my experience, it is very hard to fix typings without introducing a breaking change, so I just want to make sure we come up with the best typings possible for Redux v4. Otherwise, we would have to wait forever for the next major.


⚠ Bofrost.de has degraded performance

In 6c7de7e, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 976 ms

Resolved: Bofrost.de performance has improved in 95af656.


Book 2, version 2022-07-27, section 6.7

[KR21a] M. Khan and H. Rue. "The Bayesian Learning Rule". In: (2022). arXiv: 2107.04562.
page 265, line 30: in this section -> In this section
page 266, line 28: $\exp(R(\boldsymbol{\theta}))$ -> $\exp(-R(\boldsymbol{\theta}))$
page 268, line 18: In Section 7.3 we showed -> In Section 7.3, we show
page 268, line 46: (We discuss such algorithms in more detail in Chapter 6.): This is Chapter 6, and I don't believe the algorithms are discussed in more detail elsewhere.
page 272, line 9: $\log(p_d / (1 - p_d)$ -> $\log(p_d / (1 - p_d))$, also the log odds does not have a $1/2$.

fixed


Improved /map application

The application at http://epsg.io/5514/map should be rewritten into OpenLayers and further improved.
Inspiration from http://twcc.fr/en/ is welcome.
We should have it in two instances, one at http://epsg.io/map (a general one - allowing selection of the coordinate system of choice)
And another to show known coordinates in advance, such as http://epsg.io/5514/map or http://epsg.io/5514-5239/map (with transformation selected).
It should contain features like:

Permalink for location in hash
Formatting of the numbers a la twcc - #5
Selection of the coordinate system
Responsive design (mobile optimised) done with @daliborjanak

[ ] remove lat/lon boxes
[ ] responsive layout
[ ] popup for coordinate system
[ ] change map layers

@petrsloup done. We could deploy it.

[ ] /map add 4325 as default @petrsloup
[ ] /map remove border radius on @daliborjanak


Add dependabot

Instructions for rolling this out to any other repo:

Copy this file to .github/
Commit
Merge PR
(OPTIONAL) check here to see if it's configured: https://github.com/openforcefield/status/network/updates
(OPTIONAL) evaluate and merge any PRs it opens


Support for improved client-side ergonomics for building Try values

Right now, it looks like a value of a Try subtype must be created directly, so normal error handling control flow is needed.
What about a constructor that accepts a 0-arg callable (e.g., a fully parameterised function call), which is then invoked inside try...except logic, yielding an Ok-wrapped result if the call succeeds, and an Error-wrapped caught exception in the case of failure?

In my code I have a decorator for functions that might raise exceptions and converts that into a function that returns a Result:
_Exception = TypeVar("_Exception", bound=Exception)

@expression.curry_flip(1)
def wrap_exception(
    fun: Callable[_P, _a],
    exc: type[_Exception] | tuple[type[_Exception], ...] = Exception,
) -> Callable[_P, Result[_a, _Exception]]:
    """Wrap a function that might raise an Exception in a Result monad

    Args:
        fun (Callable[P, a]):
            The function to be wrapped.
        exc (Union[Tuple[Type[Exception], ...], Type[Exception]], optional):
            The Exception types to be wrapped into the monad. Defaults to Exception.

    Returns:
        Callable[P, Result[a, Exception]]:
            The decorated function.

    Examples:
        >>> @wrap_exception(ZeroDivisionError)
        ... def inverse(x: int) -> float:
        ...     return 1 / x
        >>> t: Result[float, ZeroDivisionError] = inverse(0)
    """

    @functools.wraps(fun)
    def _wrapper(*args: _P.args, **kwargs: _P.kwargs) -> Result[_a, _Exception]:
        try:
            return Result[_a, _Exception].Ok(fun(*args, **kwargs))
        except exc as e:
            return Result[_a, _Exception].Error(e)

    return _wrapper

By default all exceptions are captured, but you can specify which exceptions should be captured. Would that solve your problem? Note that you can use this on existing functions too, such as pandas.read_excel, like safe_read_excel = wrap_exception(FileNotFoundError)(pandas.read_excel).

Hey @Hugovdberg , indeed! This looks like a good solution. Mainly, I just want control flow to be by monadic combinators and/or pattern matchig, rather than by error handling, which this will provide :)
And fantastic that the universe of caught exceptions is controllable w/ suitable default!
Thanks a lot, I'm closing this for now.


I've modify the place where I inserted code to minimize the number of files touched.

I prefer work in status.html file because that way I don't need create any variable in peer struct.

Merged. Thanks for your contribution.


Connaisseur in EKS Environment

I could see that content trust/notary server support in Aws ECS is still yet to implement.
https://github.com/aws/containers-roadmap/issues/43
So is it possible to deploy connaisseur without the above requirements or is there any workaround for this ?

hello there.
You can run connaisseur on EKS in combination with an ECS/ECR if you either run your own Notary v1 instance or use a public Notary v1 instance (like the Docker Hub one: notary.docker.io). For running your own, head to the notary repo, there's also a pull request for a helm chart in there. But since this seems rather wacky and probably means a lot of work on your part, I'd recommend using the Docker Hub one. Also to make things more clear, just because you are using the Docker Hub Notary instance, doesn't mean you can't use the ECR. They are completly seperate from eachother, so there shouldn't be any incompatibility issues.
The later comments on the issue you posted are related to Notary v2, which is inherently different from v1. For v2 the registry  (ECR) must be changed and needs to support the new format that will be introduced. But that's future talk, as Notary v2 is currently in development and can't be used.
So for now you're stuck with v1 and can use connaisseur without issue. I hope I answered your question with this, otherwise just write back. Cheers.


PoC: CPE RTA toolbar (hides rta toolbar using css approach)

This is a PoC for building a CPE toolbar for RTA functionalities such as Undo, Redo, Save, and Switching Modes by hiding the RTA toolbar via CSS approach and using RTA APIs

closing PR.comments addressed in PR https://github.com/SAP/open-ux-tools/pull/1945


WIP: Feature/add speaker device

Started to implement the speaker device.
No clue yet on how to implement starting playback on them.
Maybe if you have an idea?

I guess you may control the playback using the SpeakerStateAttributes.playback field. In the IKEA app I found the following enum constants:
public enum PlaybackState {
        IDLE,
        PLAYING,
        PAUSED,
        NEXT,
        PREVIOUS,
        BUFFERING
}

IDLE proly corresponds to the "playbackIdle" state then. Thus, the enums are look like to be serialized according to the pattern "playbackX". I will give it a try and push the changes here.

Yeah I guessed that would be the case.
Just don't know what the gateway accepts in the playback field. I suppose all music services


Missing doc version?

The version dropdown only shows the dev version, which points to https://rep.docs.pyansys.com/dev that doesn't exist anymore.

@klmcadams @RobPasMue could you maybe take a look? thanks

Let me have a look at that

Had to do a new release so that the changes were taken into account everywhere


[BUG] Installation process fails Container kibana Waiting

Describe the bug
Installation process does not complete and indicates waiting for Container Kibana
To Reproduce
Steps to reproduce the behavior:
./elastic-container.sh start
Passphrase has been reset. Proceeding.
Starting Elastic Stack network and containers
[+] Running 7/8
⠿ Network elastic-container_default       Created                                                                             0.4s
⠿ Volume "elastic-container_certs"        Created                                                                             0.0s
⠿ Volume "elastic-container_kibanadata"   Created                                                                             0.0s
⠿ Volume "elastic-container_esdata01"     Created                                                                             0.0s
⠿ Container elasticsearch-security-setup  Healthy                                                                             7.7s
⠿ Container elasticsearch                 Healthy                                                                            30.3s
⠿ Container kibana                        Waiting                                                                            35.7s
⠿ Container fleet-server                  Created                                                                             0.1s
container for service "kibana" is unhealthy
Expected behavior
Installation process terminates successful means all containers reach the status Healthy
Screenshots
If applicable, add screenshots to help explain your problem.
Checking log files:
docker-compose logs | fgrep kibana
elasticsearch-security-setup  |    creating: config/certs/kibana/
elasticsearch-security-setup  |   inflating: config/certs/kibana/kibana.crt
elasticsearch-security-setup  |   inflating: config/certs/kibana/kibana.key
elasticsearch-security-setup  | Setting kibana_system password
elasticsearch                 | {"@timestamp":"2022-10-28T12:03:24.770Z", "log.level": "INFO", "message":"loaded module [kibana]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"elastic-container"}
elasticsearch                 | {"@timestamp":"2022-10-28T12:03:34.112Z", "log.level": "INFO", "message":"adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataIndexTemplateService","elasticsearch.cluster.uuid":"RQTcZuopT8y6bewOzx83oQ","elasticsearch.node.id":"4GSTeBR4S6W4UYaYbqaEyg","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"elastic-container"}
elasticsearch                 | {"@timestamp":"2022-10-28T12:03:34.224Z", "log.level": "INFO", "message":"adding index template [.monitoring-kibana-mb] for index patterns [.monitoring-kibana-8-]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataIndexTemplateService","elasticsearch.cluster.uuid":"RQTcZuopT8y6bewOzx83oQ","elasticsearch.node.id":"4GSTeBR4S6W4UYaYbqaEyg","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"elastic-container"}
kibana                        | FATAL CLI ERROR Error: EACCES: permission denied, open '/usr/share/kibana/config/kibana.yml'
kibana                        |     at Object.openSync (node:fs:594:3)
kibana                        |     at readFileSync (node:fs:462:35)
kibana                        |     at readYaml (/usr/share/kibana/node_modules/@kbn/config/target_node/raw/read_config.js:25:69)
kibana                        |     at getConfigFromFiles (/usr/share/kibana/node_modules/@kbn/config/target_node/raw/read_config.js:57:18)
kibana                        |     at RawConfigService.loadConfig (/usr/share/kibana/node_modules/@kbn/config/target_node/raw/raw_config_service.js:56:70)
kibana                        |     at bootstrap (/usr/share/kibana/src/core/server/bootstrap.js:57:20)
kibana                        |     at Command. (/usr/share/kibana/src/cli/serve/serve.js:216:11)
kibana                        |     at Command. (/usr/share/kibana/src/cli/command.js:102:20)
kibana                        |     at Command.listener (/usr/share/kibana/node_modules/commander/index.js:370:29)
kibana                        |     at Command.emit (node:events:513:28)
kibana                        |     at Command.parseArgs (/usr/share/kibana/node_modules/commander/index.js:892:12)
kibana                        |     at Command.parse (/usr/share/kibana/node_modules/commander/index.js:642:21)
kibana                        |     at Object. (/usr/share/kibana/src/cli/cli.js:49:9)
kibana                        |     at Module._compile (node:internal/modules/cjs/loader:1126:14)
kibana                        |     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
kibana                        |     at Module.load (node:internal/modules/cjs/loader:1004:32)
kibana                        |     at Function.Module._load (node:internal/modules/cjs/loader:839:12)
kibana                        |     at Module.require (node:internal/modules/cjs/loader:1028:19)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at Module.Hook._require.Module.require (/usr/share/kibana/node_modules/require-in-the-middle/index.js:101:39)
kibana                        |     at require (node:internal/modules/cjs/helpers:102:18)
kibana                        |     at Object. (/usr/share/kibana/src/cli/dist.js:16:1)
kibana                        |     at Module._compile (node:internal/modules/cjs/loader:1126:14)
kibana                        |     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
kibana                        |     at Module.load (node:internal/modules/cjs/loader:1004:32)
kibana                        |     at Function.Module._load (node:internal/modules/cjs/loader:839:12)
kibana                        |     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
kibana                        |     at node:internal/main/run_main_module:17:47
Desktop (please complete the following information):

OS: Centos 8
Browser [e.g. chrome, safari]
Version [e.g. 22]

Linux <my_hostname> 4.18.0-394.el8.x86_64 #1 SMP Tue May 31 16:19:11 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
sudo yum install jq git curl
Last metadata expiration check: 0:53:56 ago on Fri 28 Oct 2022 11:00:32 AM UTC.
Package jq-1.6-3.el8.x86_64 is already installed.
Package git-2.31.1-2.el8.x86_64 is already installed.
Package curl-7.61.1-25.el8.x86_64 is already installed.
Dependencies resolved.
sudo yum install docker-compose-plugin
Last metadata expiration check: 0:59:40 ago on Fri 28 Oct 2022 11:00:32 AM UTC.
Package docker-compose-plugin-2.12.2-3.el8.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
Additional context
sudo systemctl restart docker after plugin installation

Thanks, @security101
We've not experienced this issue before.
This looks to be related?
kibana | FATAL CLI ERROR Error: EACCES: permission denied, open '/usr/share/kibana/config/kibana.yml'

What're your permissions on kibana.yml (including the Z switch for the SELinux context)? Mine is
ls -lZ elastic-container/kibana.yml
-rw-rw-r--. 1 variable variable unconfined_u:object_r:user_home_t:s0 441 Oct 28 16:14 elastic-container/kibana.yml

I just ran a test on CentOS 8 (stream FWIW).
$ ./elastic-container.sh start
Passphrase has been reset. Proceeding.
Starting Elastic Stack network and containers
[+] Running 8/8
 ⠿ Network elastic-container_default       Created                                                                                0.2s
 ⠿ Volume "elastic-container_certs"        Created                                                                                0.0s
 ⠿ Volume "elastic-container_esdata01"     Created                                                                                0.0s
 ⠿ Volume "elastic-container_kibanadata"   Created                                                                                0.0s
 ⠿ Container elasticsearch-security-setup  Healthy                                                                                4.2s
 ⠿ Container elasticsearch                 Healthy                                                                                27.7s
 ⠿ Container kibana                        Healthy                                                                                58.2s
 ⠿ Container fleet-server                  Started                                                                                59.5s

Attempting to enable the Detection Engine and install prebuilt Detection Rules

Kibana is up. Proceeding

Detection engine enabled. Installing prepackaged rules.

Prepackaged rules installed!

No detection rules enabled in the .env file, skipping detection rules enablement

Waiting 40 seconds for Fleet Server setup

Populating Fleet Settings

READY SET GO!

Browse to https://localhost:5601
Username: elastic
Passphrase: elastic

[variable@localhost elastic-container]$ ./elastic-container.sh status
NAME                           COMMAND                  SERVICE             STATUS              PORTS
elasticsearch                  "/bin/tini -- /usr/l…"   elasticsearch       running (healthy)   <IP_ADDRESS>:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp
fleet-server                   "/usr/bin/tini -- /u…"   fleet-server        running             <IP_ADDRESS>:8220->8220/tcp, :::8220->8220/tcp
kibana                         "/bin/tini -- /usr/l…"   kibana              running (healthy)   <IP_ADDRESS>:5601->5601/tcp, :::5601->5601/tcp
[variable@localhost elastic-container]$

Unresponsive. Reopen if persists.


Requests for multiscale features

Hello, I appreciate very much for your source codes, and I have read your paper carefully and found it very inspiring. But I would like to request you to open source or send me the multi-scale features of TCGA and Camelyon16 data used in your paper.You have only open sourced a single scale version of them in the repository. Thank you very much and I look forward to your reply!

I hope for that too!

https://drive.google.com/drive/folders/1wHyaZkpgVGSoxPpaCeUCAFGhxcS9ZZEA?usp=sharing

Thanks a lot for your kind offer!  I think it may bother you but could you please also share the 5-scale features of TCGA? Your work is indeed very inspiring to us and such good work can bring more valuable contribution to the community.

https://drive.google.com/drive/folders/1v0ZEgSIYgriYuRn_O2p0t3RXdFHtiA61?usp=sharing

Hello! We really appreciate your help and contributions! I notice that there are two different folders in the google disk (cat and fuse). Could you please tell me what they mean? And I notice that one contain 512-dimensional features and the other contain 1024-dimensional features. I checked the paper and found that you concatenated the two 512-dimensional features to get the multi-scale mixed features. But I find a big difference between your new open-sourced features and the 20x Camelyon feautres you open sourced before. Could you tell me how to use these features. We would like to perform experiments with the two magnification features in your paper respectively. Really look forward to your reply. @binli123

Regarding 'fuse' and 'cat', you can find them in the code:
https://github.com/binli123/dsmil-wsi/blob/5216150fe7a1021c94ab1a2af6c942073bd37c4f/compute_feats.py#L110-L113
The multiscale features of camelyon16 used model-v2 in this link (you can also find this in the readme)
The features directly downloaded from the script used model-v0 in the above link.
The difference is training batch size and time. model-v0: 4096 batch size for about 500 epochs (this was obtained after the paper), model-v2: 512 batch size for about 50 epochs

Thanks for your help! We used the Cam16 20-scale features directly downloaded from the script and the Cam16 20-scale features split from the multiscale features (the first 512 dimensions) to perform the training with DSMIL. We found that the former could reach a very high AUC (above 0.97) and the latter could reach AUC=0.905 which is similar to that reported in the paper. It is possible that I made some mistakes in my operation, but I did not find the problem. Have you ever encountered this situation before? By the way, I split the training dataset and the testing dataset according to the path. We will use the former features to perform further experiments. Really thank you for your contributions! @binli123

I think I have explained it in my previous answer that the multiscale feature were computed using model-v2 and the 20x feature downloaded from the script were computed using model-v0 and they were trained using different settings and for different times.


Update ReadMe.md with pre-requisite install instructions for Windows users

The application did not work out of the box for me on a Windows 10 machine - errored saying that I needed node-gyp.
There are some useful pre-req instructions on the node-gyp repo here:
https://github.com/nodejs/node-gyp
Propose updating the ReadMe.md with some simple extra instructions for developers with Windows 10 machines.

Closed by PR #3


[Bug] Failed to start BE with JDK 17.

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
Version: master
What's Wrong?
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at<PHONE_NUMBER> (unix time) try "date -d @1712888611" if you are using GNU date ***
*** Current BE git commitID: fabd046 ***
*** SIGSEGV address not mapped to object (@0x180000) received by PID 48465 (TID 48492 OR 0x7f89cd3ff700) from PID 1572864; stack trace: ***
DEBUG   2024-04-12 02:23:31,145 196     org.apache.hadoop.security.Groups       [main]   Creating new Groups object
DEBUG   2024-04-12 02:23:31,146 197     org.apache.hadoop.util.NativeCodeLoader [main]  Trying to load the custom-built native-hadoop library...
OpenJDK 64-Bit Server VM warning: You have loaded library /doris/output/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
 3# 0x00007F8A43516400 in /lib64/libc.so.6
 4# je_free_default at /doris/thirdparty/src/jemalloc-5.3.0/doris_build/../src/jemalloc.c:3014
 5# __pthread_create_2_1 in /lib64/libpthread.so.0
 6# os::create_thread(Thread*, os::ThreadType, unsigned long) in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
 7# CompilerThread::CompilerThread(CompileQueue*, CompilerCounters*) in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
 8# CompileBroker::make_thread(CompileBroker::ThreadType, _jobject*, CompileQueue*, AbstractCompiler*, JavaThread*) [clone .constprop.0] in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
 9# CompileBroker::possibly_add_compiler_threads(JavaThread*) in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
10# CompileBroker::compiler_thread_loop() in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
11# JavaThread::thread_main_inner() in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
12# Thread::call_run() in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
13# thread_native_entry(Thread*) in /usr/lib/jvm/jdk-17.0.2/lib/server/libjvm.so
14# start_thread in /lib64/libpthread.so.0
15# clone in /lib64/libc.so.6

What You Expected?
Start BE with JDK 17 successfully.
How to Reproduce?
Use Docker development environment to build the project and start BE in the container.
docker pull apache/doris:build-env-ldb-toolchain-latest

Anything Else?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I have the same issue:
https://github.com/apache/doris/issues/33099


Signer loop: exit when shutdown signal received during upgrade loop

This PR makes the signer upgrade loop exit in case the shutdown signal is received.
The signer loop wrapper method (Greenlight::run_forever) has 2 parts, the upgrade loop (a) and the actual signer loop (b).
Inside it, the upgrade loop (a) has the following logic:

try to upgrade the signer with scheduler.maybe_upgrade

if maybe_upgrade fails due to a connection issue (error code Code::Unavailable when connecting to the scheduler), wait 3 seconds and retry
if maybe_upgrade fails due to any other reason, exit with an error
if it succeeds (e.g. returns Ok), then this loop exits and the signer loop (b) starts

The signer loop (b) is already listening and reacting to the shutdown signal.
However, the upgrade loop (a) which is just above it, detailed above, doesn't. This means that, in a situation where the scheduler is not reachable, the upgrade loop (a) would run forever, even if the shutdown signal is received.
This PR wraps  scheduler.maybe_upgrade in a tokio::select that also listens to the shutdown signal. If received, the entire signer loop is exited.

Superseded by #936


predict call for regression

Double-check if it's reasonable to return glm predictive when calling predict on regression objects (in line with torch convention AFAIK) or if we should instead incorporate observational noise (typically shown on plots)
I'll have a look at this next week but cc @Rockdeldiablo feel free to continue discussion from teams below

mah. if i had  a black box and i wanted to have an idea of where it is uncertain i would not omit the contribution of neither epistemic or aleatoric uncertainty because they signal 2 different things: the first where the nn lack data ( and a researcher may be interested to know it), the second is the intrinsic stochasticity of the measurements. For example, if you had a gap in the data right in the middle , with only the aleatoric uncertainty  the nn will give overconfident predictions to the user and in that case the "trustworthyness" is lost. In instead i use only epistemic uncertainty i will have overconfident measures where there are a lot of data points.


feat: add more GHA tips and tricks and examples

This reorganizes the pages a bit, adds the composite action example from the Scientific-Python developer summit, as well as several other new basic and advanced topics.

That was fast, thanks!


[Feature] Can I use this repo to upload multiple files to one web page?

Is there an existing issue for this?

[X] I have searched the existing issues

Is your feature request related to a problem? Please describe.
Hello:
I created one facebook page, and I can use creator studio to upload multiple files at one shot. If you upload files one by one, then quite often you can upload only a few files (less than 5), but if you use creator studio, you can upload more than 50 files at one shot. I tested with 48 files in one shot, and it works.
I want to know if I can use this repo to upload a number of files inside one folder, like: C:\Uploads.
To make things simple, if I put 2 files in the folder: C:\Uploads1.mp4 and C:\Uploads\2.mp4.
Inside my facebook page, I can go to this URL to access creator studio:
https://business.facebook.com/creatorstudio/home?reference=search_facebook
Then click on upload button and search for the folder where the files to be uploaded are saved, then Ctrl+A to select them all, then Ctrl+V to paste them in creator studio, then all the files will be uploaded.
Let me know know how I can use this repo to upload the 2 files in this folder to my facebook page.
If you have gamil account, you can register facebook without any issue, and creating a facebook is also very easy, if you don’t have mp4 format files, then you can upload other format files, like: JPG, PNG, GIF, MP3 etc..
Please advise,
Thanks,
Describe the solution you'd like
The solution as in facebook creator studio requires select mutliple files by hand.
I want to use the repo with API code, so I don't have to select multiple files by hand.
Describe alternatives you've considered
I have tried to use Playwright, but it has issue to recognize file format in Windows.  It seems it could work in Linux, but I don't work with Linux environment.

Hi, sorry this issue tracker is for bug reports only, please ask questions like this on stack overflow.


Feature/srijaa0001

I wrote function to check the number is even or odd

I wrote code to check the even and odd numbers

I wrote code to check weather the given number is even or odd


[youtube] Incorrect metadata in SDR vp9 formats in some HDR videos

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2<IP_ADDRESS>. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
No response
Description
Some HDR youtube videos have incorrect metadata when downloaded in SDR vp9 format, causing issues in VLC (washed out colors). This is a known VLC bug caused by HDR to SDR tonemapping issues but it isnt supposed to happen with SDR video files.
Its caused by two metadata entries which are not supposed to be present in a SDR video and causing VLC to freak out. The entries are "Maximum Content Light Level : 1100" and "Maximum Frame-Average Light Level : 180".
It can be fixed by removing the above mentioned metadata entries using MKVToolNix (maybe its possible using ffmpeg but I don't know of any way). The command I used was mkvpropedit -e track:1 -d max-frame-light -d max-content-light <file>
I also noticed this issue is present in some youtube videos while not in others. For example the video COSTA RICA IN 4K 60fps HDR (ULTRA HD) when downloaded in vp9 SDR format has the incorrect HDR metadata but The World in HDR in 4K (ULTRA HD) doesn't. Both are HDR videos. Also I couldn't find this issue in mp4 videos (atleast the ones I tested)
Is this a youtube issue or a yt-dlp? Since the videos are encoded on server-side and yt-dlp just downloads and merges I guess the problem is on youtube's side. I would have never even noticed this issue if vlc didn't have tonemapping issues.
Verbose log
[debug] Command-line config: ['-Uv', '-f303+251', 'https://www.youtube.com/watch?v=LXb3EKWsInQ']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2<IP_ADDRESS>
[debug] Lazy loading extractors is disabled
[debug] Python version 3.9.7 (CPython 64bit) - Linux-5.15.3-lqx2-1-lqx-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=LXb3EKWsInQ
[youtube] LXb3EKWsInQ: Downloading webpage
[youtube] LXb3EKWsInQ: Downloading android player API JSON
[debug] Loading youtube-nsig.10df06bb from cache
[debug] [youtube] Decrypted nsig g9IXL35Rr8MbgenP => lsLx25wmq7mRdQ
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr,vbr, abr, asr, vext, aext, hasaud, id
[info] LXb3EKWsInQ: Downloading 1 format(s): 303+251
[debug] Invoking downloader on "https://r3---sn-5aap5ojx-jj0s.googlevideo.com/videoplayback?expire=1637774245&ei=RR-eYenJLY-I1AbdpZa4CQ&ip=<IP_ADDRESS>&id=o-AA9Ua5mOAqWxLHUoD3gs-_VJNrRn64Ljw8qvcoPJK1WI&itag=303&source=youtube&requiressl=yes&mh=NH&mm=31%2C29&mn=sn-5aap5ojx-jj0s%2Csn-h557snzr&ms=au%2Crdu&mv=m&mvi=3&pcm2cms=yes&pl=24&initcwndbps=1165000&vprv=1&mime=video%2Fwebm&gir=yes&clen=134710778&dur=313.780&lmt=1628193060632942&mt=1637752145&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=1532434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAMtvMNWyY8qNftzg7TgPtYaeaynxH26jI-yJhImh5Iv1AiEA_NaBq-DPgQFtjzQvoDJ-ecM9hGVcFfvNDydK2DDaD_o%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgDXiiGvxv8c68kuY6TehM8pSYG76DLa_vQvx25lRGQb8CIBxxF5mcZUczQxUiqfFj4FOaXUfm09UkdVXK8bzFtXRh"
[download] Destination: COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f303.webm
[download] 100% of 128.47MiB in 00:50
[debug] Invoking downloader on "https://r3---sn-5aap5ojx-jj0s.googlevideo.com/videoplayback?expire=1637774245&ei=RR-eYenJLY-I1AbdpZa4CQ&ip=<IP_ADDRESS>&id=o-AA9Ua5mOAqWxLHUoD3gs-_VJNrRn64Ljw8qvcoPJK1WI&itag=251&source=youtube&requiressl=yes&mh=NH&mm=31%2C29&mn=sn-5aap5ojx-jj0s%2Csn-h557snzr&ms=au%2Crdu&mv=m&mvi=3&pcm2cms=yes&pl=24&initcwndbps=1165000&vprv=1&mime=audio%2Fwebm&gir=yes&clen=5199784&dur=313.801&lmt=1625670330797558&mt=1637752145&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5511222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAKP47Ez_LMMOAErRUeHDorWakbGrk4NjrocppJRtZC-CAiEAn5t7jzL46ZsFWPPMv5MBK9p58rYnzHrzlvjxXCONjAo%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgDXiiGvxv8c68kuY6TehM8pSYG76DLa_vQvx25lRGQb8CIBxxF5mcZUczQxUiqfFj4FOaXUfm09UkdVXK8bzFtXRh"
[download] Destination: COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f251.webm
[download] 100% of 4.96MiB in 00:01
[Merger] Merging formats into "COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f303.webm' -i 'file:COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f251.webm' -c copy -map 0:v:0 -map 1:a:0 'file:COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].temp.webm'
Deleting original file COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f251.webm (pass -k to keep)
Deleting original file COSTA RICA IN 4K 60fps HDR (ULTRA HD) [LXb3EKWsInQ].f303.webm (pass -k to keep)

We simply download the video as-is given by youtube. You can make sure ffmpeg is not messing anything up during merge by using just -f 303. It is either issue with youtube or with VLC


Add subtitles

Hi!
is it possible to update Ghosler with a checkbox to include the Subtitle?
https://forum.ghost.org/t/subtitles-in-newsletters-ghost-changelog/47949
Thanks

Hey @viriatusX,
Thanks for the feature suggestion.
This doesn't look too trivial to implement.
I'll bump my ghost version and take a look at how this is styled and implemented. Due to work, I cannot guarantee the ETAs on this though.

Completed via #59 🎉.


PRs now build the site and upload it for review

Hmm ... it didn't start the action. maybe "[skip ci]" is preventing it?

It triggered for the first commit. I made some minor naming changes in the second, didn't think it was worth the effort to rebuild.

amended commit to enable ci


Compatibility of Axis M5000-G PTZ Camera

Hello all.
I am planning to buy Axis Communications M5000-G 15MP PTZ Network Dome Camera. I want to programmatically control the pan, tilt, zoom movements of the camera using languages like java-script or python. That is, I want to send commands to the camera to pan to a certain location, tilt to a certain angle or zoom to a certain extent without any hardware dependency. The datasheet for the camera states that it is compliant with ONVIF profiles G, M, S, T.
I want to make sure that this repository supports the camera movement before I purchase it.
Thanks in advance.

Hello all.
I am planning to buy Axis Communications M5000-G 15MP PTZ Network Dome Camera. I want to programmatically control the pan, tilt, zoom movements of the camera using languages like java-script or python. That is, I want to send commands to the camera to pan to a certain location, tilt to a certain angle or zoom to a certain extent without any hardware dependency. The datasheet for the camera states that it is compliant with ONVIF profiles G, M, S, T.
I want to make sure that this repository supports the camera movement before I purchase it.
Thanks in advance.

Hi sirish,
I just gonna ask you if you were successful in controlling such PTZ Cams? Could you give me a hint on roadmap to do so, please.

Hello @me-shahbazi.
Unfortunately, I couldn't find time to work on this project. Hence, I cannot comment on this.