Designer cannot load PDFs in array/buffer

Describe the bug
When attempting to load a template with a base PDF in the form of a Uint8Array or ArrayBuffer, pdfme seems to wrongly think this is a base64 string and will attempt to parse it as such resulting in an error saying <some minified name>.split is not a function (full error below). Only solution is to convert the data into a base64 string.
To Reproduce
I am using the following function to load the template:
  function loadBasePDF(e: Event) {
    const file = (e.target! as HTMLInputElement).files![0];
    if (!file) return;

    const reader = new FileReader();
    reader.onload = () => {
      const a = new Uint8Array(reader.result as ArrayBuffer);
      // Not including the next line and instead passing the array directly (either as an array or a buffer)
      // will result in the attached error. Converting it to base64 first solves this issue.
      // Obviously, loading the data directly as base64 using readAsDataURL works too.
      const b64 = 'data:application/pdf;base64,' + btoa([...a].map(b => String.fromCharCode(b)).join(''));
      designer.value?.updateTemplate({
        basePdf: b64,
        schemas: [],
      })
    };
    reader.readAsArrayBuffer(file);
  }

Expected behavior
PDFMe should be able to parse PDFs in Uint8Array or ArrayBuffer form as well, but it can only parse base64 without errors.
Your Environment
- pdfme package(@pdfme/generator or @pdfme/ui): @pdfme/ui
- pdfme version: 5.2.1
- Operating system: Windows
- Browser name & version: Edge 131.0.2903.36

Your Error Log
Uncaught (in promise) TypeError: e10.split is not a function
    at kj (index.es.js:9008:15)
    at Lue (index.es.js:26766:13)
    at Nue (index.es.js:26781:33)
    at async index.es.js:130089:15

Which happens on this line:
const t = e.split(";base64,")[1] ? e.split(";base64,")[1] : e, r = Z2e(t), n = new Uint8Array(r.length);

Additional context
No response

I was handled by @necessarylion .
Released as 5.2.11


Protocol "https" not supported or disabled in libcurl

I am trying to cross-compile VLCKit for iOS, and I have this error when building rav1e. I first thought it was a cargo issue, but it seems like it's running this cargo capi extension that they don't know about. I have been tracking this error for a few days, trying to find somebody who may know where it comes from :innocent:. Could it come from rav1e, or a dependency of rav1e?
Problem
When cross-compiling VLCKit for iOS (aarch64, though armv7 seems to work), I get the following error:
touch .cargo
cd rav1e && export TARGET_CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang && export TARGET_AR=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar && export TARGET_CFLAGS="-mios-version-min=9.0 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.0.sdk -Werror=partial-availability -fno-stack-check -I/Users/jones/Documents/px4/VLCKit/libvlc/vlc/build-iphoneos-arm64/contrib/arm64-iphoneos14.0/include -g -O2" && export RUSTFLAGS="-C lto -C panic=abort -C opt-level=z" && export RUSTUP_USE_HYPER=1 && . /Users/jones/Documents/px4/VLCKit/libvlc/vlc/contrib/bin/.cargo/env && RUSTUP_HOME=/Users/jones/Documents/px4/VLCKit/libvlc/vlc/contrib/bin/.rustup CARGO_HOME=/Users/jones/Documents/px4/VLCKit/libvlc/vlc/contrib/bin/.cargo cargo capi install --target=aarch64-apple-ios --prefix=/Users/jones/Documents/px4/VLCKit/libvlc/vlc/build-iphoneos-arm64/contrib/arm64-iphoneos14.0 --library-type staticlib --release --locked --no-default-features --features=asm
Error: CliError { error: Some(failed to download from `https://crates.io/api/v1/crates/time/0.1.44/download`

Caused by:
    [1] Unsupported protocol (Protocol "https" not supported or disabled in libcurl)), exit_code: 101 }

As can be seen above, the command that fails is: cargo capi install --target=aarch64-apple-ios --prefix=/Users/jones/Documents/px4/VLCKit/libvlc/vlc/build-iphoneos-arm64/contrib/arm64-iphoneos14.0 --library-type staticlib --release --locked --no-default-features --features=asm

Steps

Clone VLCKit (https://code.videolan.org/videolan/VLCKit)
Run the script: ./compileAndBuildVLCKit.sh -a aarch64 -v. May need some patches as reported in my report here.
Observe that the error happens

Notes
Running on macOS Catalina 10.15.7

You seem to have an incomplete libcurl around and it is being used to build the cargo crate used by cargo-c.
What does
otool -L `which cargo-cinstall`

say?

It says: error: 'cargo-cinstall': No such file or directory. Could it be the issue, or is it just that the VLCKit toolchain somehow gets it for me during the build?

It seems a problem in the VLCKit toolchain, please report the problem there.

Right, I did, but I thought I could help more than that. It seems like it happens when building rav1e (cd rav1e && ...), so I thought I would ask here. But if rav1e does not use cargo capi, I guess I'll stop here.
Thanks for the help!

Please open an issue there so I can debug it, tomorrow I'll try to reproduce locally. It looks like this:
+++ b/contrib/src/cargo/rules.mak
@@ -29,10 +29,10 @@ endif
 # installation should go in a different package
 .cargo: rustup
        cd $< && RUSTUP_INIT_SKIP_PATH_CHECK=yes \
          RUSTUP_HOME=$(RUSTUP_HOME) CARGO_HOME=$(CARGO_HOME) \
          ./rustup-init.sh --no-modify-path -y
        $(RUSTUP) default stable
        $(RUSTUP) target add $(RUST_TARGET)
        unset PKG_CONFIG_LIBDIR PKG_CONFIG_PATH CFLAGS CPPFLAGS LDFLAGS; \
-               $(CARGO) install $(CARGOC_FEATURES) cargo-c
+               $(CARGO) install $(CARGOC_FEATURES) cargo-c --features curl-sys/static-curl
        touch $@

may solve your problem

I opened it here: https://code.videolan.org/videolan/VLCKit/-/issues/438. Should we continue the discussion there?
In the meantime, I will try your patch on my macOS.

I didn't check, looks like it would require a change in cargo as well since feature forwarding isn't working for more than 1 level.


Fix various spellings in markers.js

@Tawerner Thanks for your help!

@Tawerner Thanks for your help!

You're welcome. Thank you for your work! 😃


Update understand-security-groups.md

The text of Account Operators says they cannot manage Server Operators. Later it says that Account Operators is considered a service administrator group because it can modify Server Operators. This appears to be contradictory.

#assign: @robinharwood, @Xelu86
@robinharwood @Xelu86
Can you review the proposed changes?
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Adds kopia/restic uploader type

Implements #1101 part of the #1046
Testing instructions:

Uninstall previous version of OADP
Ensure you are logged as an admin

$ oc whoami
system:admin

Remove left over CRD's:

$ oc delete crd dataprotectionapplications.oadp.openshift.io volumesnapshotrestores.datamover.oadp.openshift.io volumesnapshotbackups.datamover.oadp.openshift.io cloudstorages.oadp.openshift.io

5a. You may want to use the following patch to not install or symlink podman to docker:
$ patch -p1 < /below/patch/file/location.diff

index 94cfa11..bf250a4 100644
--- a/Makefile
+++ b/Makefile
@@ -212,10 +212,10 @@ run: manifests generate fmt vet ## Run a controller from your host.
 # related enhancements that may remove the need to manually install qemu-user-static https://bugzilla.redhat.com/show_bug.cgi?id=2061584
 DOCKER_BUILD_ARGS ?= --platform=linux/amd64
 docker-build: ## Build docker image with the manager.
-	docker build -t $(IMG) . $(DOCKER_BUILD_ARGS)
+	podman build -t $(IMG) . $(DOCKER_BUILD_ARGS)

 docker-push: ## Push docker image with the manager.
-	docker push ${IMG}
+	podman push ${IMG}

 ##@ Deployment

@@ -361,7 +361,7 @@ nullable-crds-config:

 .PHONY: bundle-build
 bundle-build: ## Build the bundle image.
-	docker build -f bundle.Dockerfile -t $(BUNDLE_IMG) . $(DOCKER_BUILD_ARGS)
+	podman build -f bundle.Dockerfile -t $(BUNDLE_IMG) . $(DOCKER_BUILD_ARGS)

 .PHONY: bundle-push
 bundle-push: ## Push the bundle image.

5b. Build and deploy OADP from this PR:
$ make build-deploy
$ make deploy-olm

Create DPA, ensure to choose restic or kopia and ensure to use unsupportedOverrides with provided veleroImageFqin

Make sure to specify the backupLocations accordingly to your cluster configuration and s3 bucket
Part of the DPA for restic:
apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
spec:
  configuration:
    nodeAgentConfig:
      enable: true
      uploaderType: restic
[...]
  unsupportedOverrides:
    veleroImageFqin: 'docker.io/sseago/velero:kopia'

And for Kopia
apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
spec:
  configuration:
    nodeAgentConfig:
      enable: true
      uploaderType: kopia
[...]
  unsupportedOverrides:
    veleroImageFqin: 'docker.io/sseago/velero:kopia'

Create Backup object to trigger backup, ensure to include defaultVolumesToFsBackup: true

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: mysql-persistent
  namespace: openshift-adp
spec:
  includedNamespaces:
  - mysql-persistent
  storageLocation: dpa-sample
  defaultVolumesToFsBackup: true
  ttl: 720h0m0s

Looks like timeout isn't being set properly with nodeAgentConfig. If you add timeout to restic, it's pulled into the velero deployment, but if you add to nodeAgentConfig it's not. Here's where you need to check for nodeAgentConfig and then restic, rather than just restic:
https://github.com/openshift/oadp-operator/blob/master/controllers/velero.go#L429-L431
	if dpa.Spec.Configuration.Restic != nil && len(dpa.Spec.Configuration.Restic.Timeout) > 0 {
		fsBackupTimeout = dpa.Spec.Configuration.Restic.Timeout
	}

Other than the timeout config bug, I was able to successfuly back up and restore the OADP mysql-persistent testing app with kopia fs backup using this PR.

@sseago nice! Thanks for testing mysql-persistent.  If mysql db was successful w/ just a regular kopia backup then we have at least a data point to investigate https://issues.redhat.com/browse/OADP-2360 w/ kopia + DM.  Thank you!

/lgtm


'week' and 'weekday' as properties to 'MayaDT' object

I think it will be good to also add week and weekday to the properties of MayaDT. It will be much more convenient to user to access it like: maya_date.week and maya_date.weekday (similar to maya_date.month and maya_date.year.
weekday will be isoweekday() of datetime object with Monday as 1 and Sunday as 7

Love this!


Unable to connect to Mongo Atlas cluster - SSL

I'm trying to connect to a mongo atlas cluster, but getting the error "No suitable servers found (serverSelectionTryOnce set)" from 2 different hosts
I'm able to connect to the cluster via mongo shell on both hosts
I get a successful response when using php file_get_contents() 'https://CLUSTER-shard-00-00-2utjw.mongodb.net:27017'
certtest.php:17:
array(15) {
  'name' =>
  string(61) "/C=US/ST=New York/L=New York/O=MongoDB, Inc./CN=*.mongodb.net"
  'subject' =>
  array(5) {
    'C' =>
    string(2) "US"
    'ST' =>
    string(8) "New York"
    'L' =>
    string(8) "New York"
    'O' =>
    string(13) "MongoDB, Inc."
    'CN' =>
    string(13) "*.mongodb.net"
  }
  'hash' =>
  string(8) "1d8caf18"
  'issuer' =>
  array(3) {
    'C' =>
    string(2) "US"
    'O' =>
    string(12) "DigiCert Inc"
    'CN' =>
    string(30) "DigiCert SHA2 Secure Server CA"
  }
  'version' =>
  int(2)
  'serialNumber' =>
  string(37) "5620855521718711109215588795519274262"
  'validFrom' =>
  string(13) "160516000000Z"
  'validTo' =>
  string(13) "180521120000Z"
  'validFrom_time_t' =>
  int(1463356800)
  'validTo_time_t' =>
  int(1526904000)
  'signatureTypeSN' =>
  string(10) "RSA-SHA256"
  'signatureTypeLN' =>
  string(23) "sha256WithRSAEncryption"
  'signatureTypeNID' =>
  int(668)
  'purposes' =>
  array(9) {
    [1] =>
    array(3) {
      [0] =>
      bool(true)
      [1] =>
      bool(false)
      [2] =>
      string(9) "sslclient"
    }
    [2] =>
    array(3) {
      [0] =>
      bool(true)
      [1] =>
      bool(false)
      [2] =>
      string(9) "sslserver"
    }
    [3] =>
    array(3) {
      [0] =>
      bool(true)
      [1] =>
      bool(false)
      [2] =>
      string(11) "nssslserver"
    }
    [4] =>
    array(3) {
      [0] =>
      bool(false)
      [1] =>
      bool(false)
      [2] =>
      string(9) "smimesign"
    }
    [5] =>
    array(3) {
      [0] =>
      bool(false)
      [1] =>
      bool(false)
      [2] =>
      string(12) "smimeencrypt"
    }
    [6] =>
    array(3) {
      [0] =>
      bool(false)
      [1] =>
      bool(false)
      [2] =>
      string(7) "crlsign"
    }
    [7] =>
    array(3) {
      [0] =>
      bool(true)
      [1] =>
      bool(true)
      [2] =>
      string(3) "any"
    }
    [8] =>
    array(3) {
      [0] =>
      bool(true)
      [1] =>
      bool(false)
      [2] =>
      string(10) "ocsphelper"
    }
    [9] =>
    array(3) {
      [0] =>
      bool(false)
      [1] =>
      bool(false)
      [2] =>
      string(13) "timestampsign"
    }
  }
  'extensions' =>
  array(9) {
    'authorityKeyIdentifier' =>
    string(66) "keyid:0F:80:61:1C:82:31:61:D5:2F:28:E7:8D:46:38:B4:2C:E1:C6:D9:E2
"
    'subjectKeyIdentifier' =>
    string(59) "E1:AE:7D:1C:E9:D8:6E:15:B2:A3:A2:7D:65:53:DE:FF:A8:10:CD:98"
    'subjectAltName' =>
    string(34) "DNS:*.mongodb.net, DNS:mongodb.net"
    'keyUsage' =>
    string(35) "Digital Signature, Key Encipherment"
    'extendedKeyUsage' =>
    string(60) "TLS Web Server Authentication, TLS Web Client Authentication"
    'crlDistributionPoints' =>
    string(120) "
Full Name:
  URI:http://crl3.digicert.com/ssca-sha2-g5.crl

Full Name:
  URI:http://crl4.digicert.com/ssca-sha2-g5.crl
"
    'certificatePolicies' =>
    string(89) "Policy: 2.16.840.1.114412.1.1
  CPS: https://www.digicert.com/CPS
Policy: <IP_ADDRESS>.2.2
"
    'authorityInfoAccess' =>
    string(112) "OCSP - URI:http://ocsp.digicert.com
CA Issuers - URI:http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt
"
    'basicConstraints' =>
    string(8) "CA:FALSE"
  }
}
certtest.php:18:
int(84)

PHP 7.0.8
mongo-php-library mongodb 1.3.1

<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
ini_set('mongodb.debug', 'stdout');

$manager = new MongoDB\Driver\Manager("mongodb://USER:PASS@CLUSTER-shard-00-00-2utjw.mongodb.net:27017,CLUSTER-shard-00-01-2utjw.mongodb.net:27017,CLUSTER-shard-00-02-2utjw.mongodb.net:27017/?ssl=true&replicaSet=CLUSTER&authSource=admin");
$command = new \MongoDB\Driver\Command(["ping" => 1]);
$cursor  = $manager->executeCommand("admin", $command);
$reply   = $cursor->toArray()[0];
var_dump($reply);

mongo.debug output - https://gist.github.com/jsha09/3200e932209c565a98bca8c31d5ef200

Apologies for never replying to this issue. I must have missed the original GitHub notification and lost track of it. Is this issue still occurring with the latest driver version?
Taking a look through the mongodb.debug trace output you shared (side note: thanks for providing that up front!), it's clear that the driver is connecting to some servers, as we see replies to isMaster commands in the logs. If I had to guess, the driver was attempting to execute the ping command against a primary server and the replica set did not have a primary available ("RsNoPrimary" appears in the logs as well). If this issue was not permanent, it's likely that you connected to Atlas while it was in the middle of a replica set election.

Closing due to inactivity. Feel free to follow up if needed.


Remove {CameraDescription/Optics/Telescope}.from_name methods

Please describe the use case that requires this feature.
While these are handy shortcuts for tests etc. it turned out to also create a lot of bugs and surprising behavior,
e.g. people analyzing / plotting data simulated with Prod5 using the from_name version, which loads an older geometry for most telescopes.
Describe the solution you'd like
We should remove these methods and make it clear that the subarray information should always be read from the input file or some other source instead of relying on the from_name mechanism.
Describe alternatives you've considered
Making the from_name system more complex by including more information like versions and prod numbers. But I don't like it, we should just remove the method instead of making the leaky abstraction more complex.

The original intent was that we would have a selectable set of input data for this and other tests, e.g. one per prod.  See #738 for example.  It just never got implemented.

@kosack I think we can better enable this through fixtures, e.g. in the pull request #1985 I introduce fixtures like subarray_prod5_paranal.

There was an issue again: #2157 we should probably go ahead with this...

So it's been a while since I commented on anything with respect to ctapipe, but since this has an effect on a number of ongoing projects with current generation instruments I'd like to make a request. I agree that getting rid of the from_name functionality makes sense as you propose @maxnoe; I think the big problem is that many users don't understand the different camera model versions. It's also not currently possible (as far as I can tell) to determine the available options for this method (i.e. what files are on the data server) or to tell where the associated camera geometry files actually came from.
Going to a purely from_url/from ctapipe-process file-type system is good (because it's more difficult to abuse it). But somewhere in the documentation I'd ask there be downloadable links to historical camera geometry files, who contributed them, and what exact instrument/simulation model they correspond to. The reality is that we're going to be dealing with (i.e.) CHEC-S on ASTRI data or H.E.S.S. data not in ctapipe-compatible data formats for some time yet.

The reality is that we're going to be dealing with (i.e.) CHEC-S on ASTRI data or H.E.S.S. data not in ctapipe-compatible data formats for some time yet.

Just to comment on this part for now: that involves IO plugins and other custom code by the Experte for those systems. These IO plugins should provide the corresponding instrument descriptions by whatever means they deem appropriate and should not rely on the ctapipe test data server to provide them.
I.e. ctapipe_io_lst bundles the latest, real LST 1 geometry and single pe pulse shape files.

@kosack I think we can better enable this through fixtures, e.g. in the pull request https://github.com/cta-observatory/ctapipe/pull/1985 I introduce fixtures like subarray_prod5_paranal.

Yes but the current fixtures are a bit of a mess, and I don't see a nice way to iterate through a set of them. E.g. I would like to be able to have test functions like:
@pytest.parametrize(dataset=['prod3','prod5b', 'LST-1'])
@pytest.parametrize(camera=['LSTCam','NectarCam'])
def test_something(dataset, camera):
     utils.set_dataset(dataset)
     geom = CameraGeometry.from_name(camera)
     ...

To test a function across all implementations of a particular instrument info.


Adding dynamic Severity

This PR adds the ability to use a dynamic severity the same way we can use dynamic custom state by using a provider instead of a fixed value.
Use case:
We used custom state before severity existed in FluentValidation to do the same.
Our validators then created either a warning or an error depending on various conditions like "Does the user have a license to use this feature?" if he does the validation resulted in a warning, if he didn't the validation resulted in an error.

If you could also rebase onto the 9.x-dev branch too that'd be great!

If you could also rebase onto the 9.x-dev branch too that'd be great!

Done.

Thanks! I'll aim to get this merged in later in the week.

Looks like there are some compile errors when retargetting to 9.0. CI didn't pick that up for some reason as azure devops was having issues and stopped picking up changes from github. I've fixed CI now, and I've pushed changes to your branch to fix the compile errors.


Dispose command line args enumerator

Fixes https://github.com/aspnet/Configuration/issues/463
@divega

:shipit:

dc569023de8db0234b39ec40f4e930f56958944c


Propagate bad query exception all the way so that it does not get logged as an error

Codecov Report

Merging #1328 into master will increase coverage by 7.75%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #1328      +/-   ##
==========================================
+ Coverage   59.11%   66.86%   +7.75%
==========================================
  Files         781      781
  Lines       38105    38107       +2
  Branches     5001     5001
==========================================
+ Hits        22524    25482    +2958
+ Misses      13988    10958    -3030
- Partials     1593     1667      +74

Impacted Files
Coverage Δ

.../com/linkedin/pinot/core/plan/CombinePlanNode.java
75% <0%> (+9.78%)
:arrow_up:

.../resources/BasePinotControllerRestletResource.java
87.23% <0%> (ø)
:arrow_up:

...linkedin/pinot/common/utils/helix/HelixHelper.java
52.66% <0%> (+0.66%)
:arrow_up:

...t/creator/impl/SegmentIndexCreationDriverImpl.java
85.56% <0%> (+0.68%)
:arrow_up:

.../helix/core/realtime/SegmentCompletionManager.java
68.31% <0%> (+0.74%)
:arrow_up:

...indexsegment/generator/SegmentGeneratorConfig.java
58.56% <0%> (+1.19%)
:arrow_up:

...rEqualitiesToInClauseFilterQueryTreeOptimizer.java
90.9% <0%> (+1.29%)
:arrow_up:

...n/pinot/core/common/datatable/DataTableImplV2.java
94.52% <0%> (+1.36%)
:arrow_up:

...not/broker/requesthandler/RangeMergeOptimizer.java
88.7% <0%> (+1.61%)
:arrow_up:

...mon/metadata/segment/OfflineSegmentZKMetadata.java
63.79% <0%> (+1.72%)
:arrow_up:

... and 195 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 681ca0d...e0847ef. Read the comment docs.


GUACAMOLE-303: Allow root directory of SFTP filesystem to be configured.

This change adds a new "sftp-root-directory" parameter to VNC, RDP, and SSH which defines the root directory exposed via Guacamole's file browser. Some care is taken to ensure that requests outside this root directory cannot be made via the file browse, in that both the root directory and the requested filename are normalized prior to being concatenated.

Looks good to me.  @jmuehlner anything you see before it gets merged to master?

Looks reasonable to me!

Testing this with guacamole-0.9.14 the feature sftp-root-directory does not appear to work with VNC. I have not tested the other protocol paths.

@swplotner If you're having issues with this please open a thread on the mailing list - this is a closed/merged PR.


Consider allowing a string response to be inferred as text(..)

Instead of saying return text(..) in a route, allow return 'my text' by inferring the response type in write_response.
sanic/server.py:
class HttpProtocol
    ...
    ...
    def write_response(self, response):
        try:
            keep_alive = self.parser.should_keep_alive() and not self.signal.stopped

            try:
                self.transport.write(response.output(self.request.version, keep_alive, self.request_timeout))

            except AttributeError:
                # attempt to send the response as a text response
                if isinstance(response, str):
                    response = text(response)
                    self.transport.write(response.output(self.request.version, keep_alive, self.request_timeout))
                else:
                    raise

            if not keep_alive:
                self.transport.close()
            else:
                self.cleanup()
        except Exception as e:
            self.bail_out("Writing request failed, connection closed {}".format(e))

I like this, on another note we should also be allowed to return a dict and it automatically converts to json

I think explicit is probably better the implicit. Does this change any of the benchmarks? If it slows things at all, may not be worth it.

@eseglem From an outside opinion, I would say that you are still allowed to do it by explicitly using a HTTPResponse. And the terseness of not having to put redundant information about the response is a huge usability feature.
Also while I don't believe the two isinstance checks will slow performance, if it is a concern there could be an initial isinstance check for HTTPResponse to short circuit the other checks
(regarding https://github.com/channelcat/sanic/pull/52)
    pass```

Personally I think it should only be explicit. It should be on the user to know what they are returning. They should know if you are returning text() or json(), and put it on the return. I do not thing it detracts from the usability at all. Even putting in the isinstance(response, HTTPResponse) first adds an additional call to isinstance which should not be needed at all. It gets added to every single return, meaning (based on the benchmarks) 30,000 calls to isinstance / second, which just seems like unnecessary overhead for something that should already be known by the user. And if they do not know what type it is, they can do an if before the return. Just personally it seems like an unnecessary feature when the number one purpose of the system (to me at least) is to go fast. Anything that slows the system down, even by a tiny fraction, isn't worth it to me.
I may be a bit overly concerned. Based on a bit of timeit, it may only take a 0.2% hit in performance. Which isn't really that much, but I am afraid of 100 little 0.2% hits adding up to a massive hit.

@eseglem Good points. I certainly can't argue with explicit over implicit. Especially with the main goal of goin' fast, anything that user can declare that the code doesn't have to compute is a definite win.

How often are you using raw text or html response vs json or template in real projects? I don't think we need any sugar foo so rare cases.

Maybe a bit more descriptive error message could be added to make Sanic a bit more beginner friendly? If you're coming from flask with only the current error it's kinda hard to figure out.

@channelcat @seemethere are we still considering this? It seems like it would be a great usability enhancement. Related, since performance seems to be delaying this: how exactly should we run benchmarks? I'll add it to the readme somewhere if you can point me to the instructions...

I think this one is done. Will re-open if there's a more performant way to do it.


Identity And Other Logical Operators

I often find my self having to stream transformation only if a certain condition is met so it ends up looking something like this:
var xs = [1, 2, 3, 4, 5]

_.seq(
    _.through(function (xs) {
        if (req.query.count == null) {
            return xs;
        }
        return xs.take(count)
    }),

    _.toArray(function (xs) {
        console.log(xs);
    })
)(xs);

That's quite clunky so what I actually do is:
_.seq(

    req.query.count == null ? require('ramda').identity : _.take(count),

    _.toArray(function (xs) {
        console.log(xs);
    })
)(xs);

I think it would be nice to include an identity function in the operators section like with _.not. And while we're at it wouldn't do any harm to add _.and and _.or. What do people think? I could have a PR done for this very quickly.

I'm against this. There's no reason to include these combinators in Highland, they're nothing to do with streaming. What's wrong with using Ramda like you're doing?

Nothing wrong with using Ramda at all but just because I don't mind that dependency doesn't mean others will. How come we have _.not and _.add then? They're not really anything to do with streaming either but they're very useful from a functional programming point of view, which I thought was one of the areas Highland was exploring.

If I'd been involved with Highland when not was added I'd have protested then as well :smile:

Hehe. It's a good point though, I think we either have all the common operators or none.

If it's already composable, why one should copy this stuff into the Highland source tree? Focus on keeping things DRY.

I think that once we will all agree on highland's extensibility mechanism we will maybe need to cleanup what belongs and what does not belong to the core.

@jeromew, great point. :+1:

This escalated quickly! :) I actually agree with @quarterto, the operators probably don't belong here. I just thought there was a reason I didn't understand as to why they were there and wanted to add the 'missing' ones for completeness.
That said, I still think the if/else functionality for applying a tranform is useful. Is there a way of doing it using the chained syntax without that clunky through?

Beyond that, we should rework the docs as currently they state:

Re-thinking the JavaScript utility belt

Which links to lodash, underscore, and async.
Also, the docs alias highland to _ which is usually the variable used by lodash and underscore; which can lead to further confusion.

well I think that @caolan had the idea of 'rethinking the javascript utility belt' ;-) and maybe show that with highland, _ could replace those 3 libraries.
I'll leave any change on this sentence to him ;-)
@svozza I haven't tried it and am not familiar with it but I think the if/else can be done via a stream redirection next(stream)

Yes, it did appear @caolan intended for highland to replace lodash, etc. It would be awesome to include examples of how to extend highland to use methods from lodash, like:
var highland = require('highland');
var lodash = require('lodash');
var _ = highland.extend(highland, lodash);

// based on example from lodash docs
_.intersection([1, 2, 3], [5, 2, 1, 4], [2, 1])
.toArray(function(result) {
  // → [1, 2]
});

@svozza I don't think there's a way to do it using the chained syntax without through, but I don't agree that it's clunky.
_([1, 2, 3, 4])
    .through(req.query.count == null ? require('ramda').identity : _.take(count))
    .toArray(_.log)

This looks exactly like the code you went with, except with an extraneous through.

Yeah, I was overcomplicating it before, that's just as nice a way of doing it.


Fix gitignore to include Vagrant generated files

Pull Request #1540 added the Vagrant build to App Inventor but overlooked to include the Vagrant generated files to the .gitignore file. We don't want contributors to add these files to the sources by accident.

Can one of the admins verify this patch?


expand command and directory before executing

I made a surgical change targeted only this scenario.

:shipit:


Malformed declaration crashes flow server without error

If I put this in my underscore declarations file:
declare module "underscore" {
  declare function any<T>(list: Array<T>, pred: (T)=>boolean): boolean;
}

then running flow check . produces a helpful error:
$ flow check .
/Users/danvk/github/pileup.js/lib/underscore.js:16:52,53: Unexpected token =>

Found 1 error
Fatal error: exception Failure("Can't read library definitions file /Users/danvk/github/pileup.js/lib/underscore.js, exiting.")

However, running flow status . does not:
$ flow status .
Error: flow server is busyzing. If it was just started this can take some time. Retrying... \
Flow server launched for /Users/danvk/github/pileup.js
Spawned flow server (child pid=41846)
Logs will go to /var/folders/9q/340_463n2fvbzjg232s3hrt00000gn/T/flow_danvk/danvk-zSUserszSdanvkzSgithubzSpileup.js.log
The flow server will be ready in a moment.
Error: flow server is busyzing. If it was just started this can take some time. Retrying... -
Flow server launched for /Users/danvk/github/pileup.js this can take some time. Retrying... \
Spawned flow server (child pid=41889)
Logs will go to /var/folders/9q/340_463n2fvbzjg232s3hrt00000gn/T/flow_danvk/danvk-zSUserszSdanvkzSgithubzSpileup.js.log
The flow server will be ready in a moment.
Error: flow server is busy
Out of retries, exiting!lizing. If it was just started this can take some time. Retrying... /

This is especially confusing when using the editor integration, since no errors ever appear.
I'm using flow 0.6.0.

BTW, that declaration should be:
  declare function any<T>(list: Array<T>, pred: (el: T)=>boolean): boolean;

(i.e add the el: ).

Hi Dan - appreciate the repro! Fix should be in the next deploy.

0.13 is much more informative when declarations are invalid. Please reopen if you continue to have issues after upgrading.


Default prometheus to false

https://github.com/crowdsecurity/cs-firewall-bouncer/blob/5afbe527ad361e578fb1bdf523a2c34c31258149/config/crowdsec-firewall-bouncer.yaml#L55-L56
Should we default to false to not potentially use a port for no reason if the user doesnt know. If they want to be able to get metrics then they can simply come in turn to true and restart service.

done


mobile: pull in subsequent work from the old repo

OTT
Think I got everything except the version bumps and most android/ios project file changes which looked innocuous.
Tested what I could on simulator against a local ship.
Fixes LAND-1646

Conflicts should evaporate after #3310 lands


Feature request: automatically generate inheritance graph/dependency graph

Prerequisite
Please try out this onboarding document (Step 1 to 4) to install and get familiar with axs.
Steps
This is a clone of Axs main kernel, we can play with it here and prepare a pull request for it when it is ready. Basically, we would like to:
Step 1. Generate inheritance graph for axs
Based on the python script (https://github.com/elimkwan/axs_clone/blob/graph/draw_hierarchy.py and https://github.com/elimkwan/axs_clone/blob/graph/gv_test.py) given by Leo, we can rework it into something. The current scripts work when placed in ~/axs, then `` For a given entry, it can show the parent(s) and the parent(s) of the output entry (if there is a output entry). For example in https://github.com/krai/axs/blob/master/core_collection/bert_squad_onnxruntime_py/data_axs.json, the parent is python_script (because https://github.com/krai/axs/blob/master/core_collection/bert_squad_onnxruntime_py/data_axs.json#L2) and the parent of the output entry is base_bert_experiment(because https://github.com/krai/axs/blob/master/core_collection/bert_squad_onnxruntime_py/data_axs.json#L63). This observation is right most of the times: When you see _parent_entries at the first level of the json, the list gives the parents of the current entry; when you see _parent_entries at the deeper level of the json, the list gives the parents of the output. Ideally, for a given entry, we should be able to track who is the parents, and who are the parents of the parents etc etc.
For example, by calling the script
python draw_hierarchy.py {path to bert_squad_onnxruntime.py}

We can get something like this:

There are many different ways to visualize it better (graphviz is widely supported), open to suggestions!
Step 2 Generate inheritance graph with more details
For a given repo/entry, it can show the parent(s) and the parent(s) of the output entry (if there is a output entry).
And it can show the functions in their code_axs.py
Step 3. Generate dependency graph
Include dependency as well, not just capturing the parent-child relationships. Similar to something like this drawio diagram for x-qaic-private.
Step 4. Integrate into kernels
Achieving something like this:
axs showgraph {repo-name}
axs showgraph {target-entry-name}

Update:
At the moment, the graph for bert_squad_onnxruntime.py looks something like this,

Next Steps :
Issues:

How to access nested json object keys and values? Like, the example shown below,(_parent_entries from output_file_path)

"output_file_path": [ "^^", "execute", [[
        [ "get", "__record_entry__" ],
        [ "attach", [ "^", "work_collection" ] ],
        [ "plant", [ "^^", "substitute", [[
            "_parent_entries", [ "AS^IS", "AS^IS", [ "^", "byname", "base_bert_experiment" ] ],
            "tags", [ "program_output", "bert_squad" ],
            "model_name", "#{model_name}#",
            "framework", "#{framework}#",
            "output_file_name", "#{output_file_name}#",
            "desired_python_version", "#{desired_python_version}#"
         ]] ] ],
         [ "save" ],
         [ "get_path_from", "output_file_name" ]
      ]] ],
      ```
2. How to get the `_parent_entries` under `_parent_entries` i.e. to retrieve multiples parents from the `data_axs.json` file.

Hi! To answer your question, let the current entry be A:
To find A's parents, we can just use a in-built functions. This will return the list of A parents:
parents = target_entry.get("_parent_entries")

This line currently forced us to only focus on 1 parent. Relaxing it will allow us to consider the multiple parents case.
To find A's output's parents, we need to use regex, like the example we coded up last time
https://github.com/elimkwan/axs_clone/blob/master/core_collection/graph/code_axs.py#L117
obj = ['_parent_entries', ['AS^IS', 'AS^IS', ['^', 'byname', 'base_bert_experiment']], 'tags', ['program_output', 'bert_squad'], 'model_name', '#{model_name}#', 'framework', '#{framework}#', 'output_file_name', '#{output_file_name}#', 'desired_python_version', '#{desired_python_version}#']
    item = find_key(obj, "byname")
    ..
key == "_parent_entries" and re.search(r"^\[(?:'|\")_parent_entries(.*)", str(obj)):

This regex syntax only find the first parent, please check on its documentation on how to expand it. There are 3 cases:

When A's output has 1 parents

"_parent_entries", [["^", "byname", "base_bert_loadgen_experiment"]],

When A's output has 2 parents

"_parent_entries", [["^", "byname", "base_bert_loadgen_experiment"], ["^", "byname", "base_qaic_experiment"]],

When A's output has 2 parents  with delays AS^IS (for delaying intepretation)

"_parent_entries", ["AS^IS" , "AS^IS", [["^", "byname", "base_bert_loadgen_experiment"], ["^", "byname", "base_qaic_experiment"]]],

Note that when copying examples from axs data json, they are in double quote", as shown above, but after loading it into python and printing it out, they become single quote '.
@sahelib25 Please look into how to expand the graph entry to handle the case when:

A has multiple parents
when any of A's parents has multiple parents
when A's output's parent has the delay sign in front of it
when A's output has multiple parents

Thanks!

Hi @sahelib25 , functionality wise it seems great!
Next step:

making it visually appealing
test it works for all axs entry (If not all, at least most of them)
sync with current axs kernals


questions about code

I wonder if the code in https://github.com/liusongxiang/efficient_tts/blob/d186a56bf87e2c688158179f0f41b981718aebdb/nntts/models/efficient_tts.py#L338 is correct？It seems two tensors with different size make subtraction，[B,T2,1] and [B,T1,1]

imv.unsqueeze(1) has shape of [B, 1, T2] and p.unsqueeze(-1) has shape of [B, T1, 1].
The minus operation will conduct broadcast first, which means the result will have shape of [B, T1, T2].

got it, thank you!


Backspace does not work in config text boxes (Web UI)

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
**Home Assistant release (hass --version):
0.59.2
**Python release (python3 --version):
Python 3.4.2
Component/platform:
Web UI - text boxes
Description of problem:
Using latest Chrome on macos (did not test other browsers yet): when inputting text into the configuration boxes (i.e. on the States page, or anywhere a text input exists) the Backspace (labeled delete on some keyboards, upper right over the backslash key) key does not perform its function of deleting the character to the left of the cursor.
Expected:
The character to the left of the cursor should be deleted, which is standard Backspace functionality. A workaround is to use the forward-Delete key (of the Insert/Delete duo).
Problem-relevant configuration.yaml entries and steps to reproduce:

Click in text box on config page.
Type something.
Make a mistake.
Hit backspace to fix the error.
Cry profusely at the inconvenience of having to use the forward Delete key.

Traceback (if applicable):
n/a
Additional info:
This is probably the result of a bug in or misconfiguration of the library used for fancy text boxes, just my guess. No errors exist in the JavaScript console. If I have time to dig into the code I can send over a pull request but I wanted to report my woes first.

Works fine here (chrome on android and Windows 10), could it be because you're using an old python version (should be on 3.6)?


cleaup: fix command in the error msg

Describe what this PR does
This PR fixes typo in err msg:
Missing GO_ARCH argument for building image, install Golang or run: make image-cephcsi GOARCH=amd64

/test commitlint

/test commitlint


[Docs] Use PyTorch style docs theme

Motivation
OpenMMLab docs will change to use PyTorch style, refers to https://github.com/open-mmlab/mmcv/pull/1321
Modification
Update docs conf according to mmcv version.
BC-breaking (Optional)
No, just docs.
Use cases (Optional)
No
Checklist
Before PR:

[x] Pre-commit or other linting tools are used to fix the potential lint issues.
[x] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
[x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
[x] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

[x] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
[x] CLA has been signed and all committers have signed the CLA in this PR.

it seems no change to chinese docs, and english docs do change.

it seems no change to chinese docs, and english docs do change.

Well, looks like "Build pull request preview docs" won't build translation docs. I don't know how to config it. If you want to preview the Chinese docs, use this link


Added more tests for CLI commands.

This PR

Adds more tests in the tests/test_cli_loads.py folder to test the other commands
Updates the contributing file and adds testing part in it instead of ## Tests HAHAHAHAHAHAHA good one

As asked in #7

hi @Gitstar-OC! thank you for the contribution!
in the future when opening a pull request, be sure to merge from a branch other than main even on your fork. it allows the maintainers to easily add to your contribution and help get a PR past the finish line :)
since i can't push to this branch, i'm including my changes as notes in the review. if you can commit these, i'll ask @tcdent to review again (since my work is in this PR now) and merge

Hey sorry for the problems caused I will solve them.

Update worked on the compelete part, will create a pr shortly after a few hours. The work is done

one more edit required. be sure to run tox to test locally and ensure it works

All of the things are completed, also ran the tests. Sorry that you faced some problems becuase I did not create a new branch will create a new one for other issues, could I get to work a little bit on frontend if that's possible ?

Tests were successful

I don't understand why are this workflows failing ? They are working if I run the tests in the development mode and failing in production.
I will check the error and fix them don't worry.


Update event-hubs-java-get-started-send.md

Updating code to reflect the most recent java client. This will need to be changed again once 1.0.0 is released.

@sabeegrewal : Thanks for your contribution to the Azure documentation! The author, @sethmanheim, has been notified to review your proposed change.

@sabeegrewal Thank you for your contribution, we appreciate it! We'll merge this right away. #sign-off


Add additional check in get for _Bit class

Type of changes

[x] Bug fix
[ ] New feature
[ ] Documentation / docstrings
[ ] Tests
[ ] Other

Checklist

[x] I've run the latest black with default args on new code.
[ ] I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
[ ] I've added tests for new code.
[x] I accept that @willmcgugan may be pedantic in the code review.

Description
Fix a bug when trying to access attributes of uninitialized Style class.
Before change:
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from rich.style import Style
>>> Style.blink
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kyle/projects/rich/rich/style.py", line 26, in __get__
    if obj._set_attributes & self.bit:
AttributeError: 'NoneType' object has no attribute '_set_attributes'

After change:
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from rich.style import Style
>>> Style.blink
>>>

Accessing an attribute of an uninstantiated class sends None as the first argument to __get__. Example:
>>> class Foo:
...     def __get__(self, instance, owner):
...             print("instance: ", instance)
...             print("owner: ", owner)
...
>>> class Bar:
...     foo = Foo()
...
>>> Bar.foo
instance:  None
owner:  <class '__main__.Bar'>
>>>

I'm not sure if this behavior is intended or not? I don't know the reasoning behind the design of this __get__ function. Also I just assumed to return None. We could also return self.bit instead.

I don't know of a situation where you would need to access that property without an instance. Do you have a use-case for this?

My use case is that I am assuming everything that is returned by dir() is an accessible attribute, but maybe I shouldn't assume that? I'm building an object explorer program, similar to how rich.inspect works but with some more features: https://github.com/kylepollina/objexplore/
Ran into an attribute error which led me down a small rabbit hole which helped me learn a little bit on how __get__ works
If you don't think that this is worth adding that's fine by me :) You can just close

Ah. I'm afraid you can't assume that. You will encounter many more examples of attributes where accessing them throws an exception.
Your object explorer looks very cool, BTW.

Thanks! It looks like you've sent some traffic over to it too ;)


Getting  Error: [$compile:nonassign] while using Angular ui-bootstrap

Hi,I am getting the following error while navigating one page to another page using ui-bootstrap of Angular.js.
Error: [$compile:nonassign] http://errors.angularjs.org/1.4.6/$compile/nonassign?p0=%24state.current.name%20%3D%3D'principal.myplanManagement.myPlan'&p1=uibTab
    at Error (native)
    at http://oditek.in/Gofasto/js/angularjs.js:6:416
    at q (http://oditek.in/Gofasto/js/angularjs.js:74:45)
    at l (http://oditek.in/Gofasto/js/angularjs.js:74:120)
    at Object.<anonymous> (http://oditek.in/Gofasto/js/angularjs.js:116:15)
    at n.$digest (http://oditek.in/Gofasto/js/angularjs.js:130:71)
    at n.$apply (http://oditek.in/Gofasto/js/angularjs.js:133:236)
    at HTMLAnchorElement.<anonymous> (http://oditek.in/Gofasto/js/angularjs.js:253:36)
    at HTMLAnchorElement.m.event.dispatch (https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js:4:8549)
    at HTMLAnchorElement.r.handle (https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js:4:5252)

I am explaining my code below.

 <uib-tabset>
 <uib-tab ui-sref="principal.myplanManagement.myTimeTable" active="$state.current.name =='principal.myplanManagement.myTimeTable'">
      <uib-tab-heading>Add MyTimeTable</uib-tab-heading>
    </uib-tab>
    <uib-tab ui-sref="principal.myplanManagement.myPlan" active="$state.current.name =='principal.myplanManagement.myPlan'">
      <uib-tab-heading>Add MyPlan</uib-tab-heading>
    </uib-tab>
    <uib-tab ui-sref="principal.myplanManagement.myWDS" active="$state.current.name=='principal.myplanManagement.myWDS'">
      <uib-tab-heading>Add MyWDS</uib-tab-heading>
    </uib-tab>
    </uib-tabset>

When i am moving to Add MyPlan page this type of error i am getting in console.Please help me to resolve this error.

@subhrajyotiw5rtc as already asked please don't open issues for general support questions but please direct them to an appropriate support channel instead: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#question
Thnx!

@pkozlowski-opensource  How can i get the appropriate support channel ? I have to fix this issue.Please help me.

How can i get the appropriate support channel

From https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#question:

If you have questions about how to use AngularJS, please direct these to the Google Group discussion list or StackOverflow. We are also available on IRC and Gitter.


Update cache-how-to-vnet to explain port requirements for georeplication

… Geo-Replication scenario
/cc @yegu-ms @JonCole

@TimLovellSmith : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Weird width on MXParallaxHeader 1.1.0

Issue Description

Weird width on MXParallaxHeader 1.1.0

Environment Information

MXParallaxHeader 1.1.0
iPhone7 11.3.1, iPhone XR 13.2.2
Xcode 11.2.1

Your Code
let height: CGFloat = Constants.APPDELEGATE.cellSize.height + 50 + UIWindow.safeAreaTopInset

    override func viewDidLoad() {
        super.viewDidLoad()

        detailTableView.contentInsetAdjustmentBehavior = .never
        headerView = PostDetailCardView()
        detailTableView.parallaxHeader.view = headerView
        detailTableView.parallaxHeader.height = height
        detailTableView.parallaxHeader.mode = .fill
        detailTableView.parallaxHeader.minimumHeight = 0
}

Screenshot

by MXParallaxHeader 0.6.1

by MXParallaxHeader 1.1.0

Same issue here :(

as for me, add width constraint of your headerView  cant fix this issue

I have the same issue

I have the same issue

Nice fix, it works perfectly for me !

Getting the same issue. The above-mentioned solution is also not working when header is instantiated from the nib.

Scroll not working after I am adding width constraint!
please help

Above solutions didn't work same error

Issue is in your latest version of pod

Same problem here and thanks to @Nahatakyan , below code fix it;

tableViewHeaderView.translatesAutoresizingMaskIntoConstraints = false
tableViewHeaderView.widthAnchor.constraint(equalTo: tableView.widthAnchor).isActive

I solved fixing width in my header view!


[Improvement] Applying blacklist to tags/keywords

I don't just mean the titles of artwork, by the way. It would seem feasible to make a GET request for each thumbnail link that populates on a webpage and leads back to the main page of the artwork submission (which it usually does), which follows the format:
https://www.furaffinity.net/view/
When it retrieves HTML data from that page, the script can then compare the keywords embedded in the HTML and check them against the user-specified blacklisted tags/filters. If there is a match, it should delete the image.

I agree, this would be great. The blacklist only works on the text in the post, but searching and blocking from the keywords would be better.

I agree, this would be great. The blacklist only works on the text in the post, but searching and blocking from the keywords would be better.


Use tmp dir as a gradle home

Right now kt2cpg fails if you don't have ~/.gradle. You can reproduce it by mv ~/.gradle ~/.gradle2

Discussed offline: we'll keep the user's default gradle home because it might contain logic necessary for resolving a project's depenencies; setting it explicitly could break it

see https://github.com/joernio/joern/pull/1218 for a fix of the currently broken functionality


Print duplicate content in a Jupyter cell

Does this issue occur when all extensions are disabled?: Yes/No
No

VS Code Version: 1.79.0
OS Version: win11

Steps to Reproduce:
start a lightgbm.train and set verbose_eval to 50, and it print the training information as following:
Training until validation scores don't improve for 400 rounds
[50]	training's r2: 0.118448	valid_1's r2: 0.117714
Training until validation scores don't improve for 400 rounds
[50]	training's r2: 0.118448	valid_1's r2: 0.117714
[100]	training's r2: 0.193728	valid_1's r2: 0.192412
Training until validation scores don't improve for 400 rounds
[50]	training's r2: 0.118448	valid_1's r2: 0.117714
[100]	training's r2: 0.193728	valid_1's r2: 0.192412
[150]	training's r2: 0.242462	valid_1's r2: 0.240691
Training until validation scores don't improve for 400 rounds
[50]	training's r2: 0.118448	valid_1's r2: 0.117714
[100]	training's r2: 0.193728	valid_1's r2: 0.192412
[150]	training's r2: 0.242462	valid_1's r2: 0.240691
[200]	training's r2: 0.27447	valid_1's r2: 0.272249

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


[HPRO-1195] Fetch IDV histories from RDR

Q
A

Target Release?
<IP_ADDRESS>

Bug fix?
❌

New feature?
✅

Database migration?
❌

New configuration?
❌

Composer updates?
❌

NPM updates?
❌

Jira Ticket(s)
HPRO-1196

Due to User workflow needs, IDV on the Participant Summary pages needs to be reverted back to using the RDR API to display historical IDV records.


Bug: Problems with "node_security_group_enable_recommended_rules = false"

Is your request related to a new offering from AWS?
No - not related to any new offering. Just some enhancement to the terraform module
Is your request related to a problem? Please describe.
I frustratingly lost a few hours of time working my way around a specific problem.
As a consumer of this terraform module, I would like to:

Get the benefit of using the node_security_group_enable_recommended_rules = true
AVOID having <IP_ADDRESS>/0 egress on the node_security_group

Context:
Since I could not allow <IP_ADDRESS>/0 egress for my nodes, I tried to replicate the recommended_rules by hand, and I set node_security_group_enable_recommended_rules  to false.
It looked like this:

resource "aws_security_group_rule" "_ingress_cluster_api" {
  description              = "ARMEN Cluster API to node"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 443
  to_port                  = 443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_egress_cluster_api" {
  description              = "ARMEN Node to Cluster API"
  type                     = "egress"
  protocol                 = "tcp"
  from_port                = 443
  to_port                  = 443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_ingress_cluster_kubelet" {
  description              = "ARMEN Cluster API to node kubelet"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 10250
  to_port                  = 10250
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_self_coredns_tcp" {
  description              = "ARMEN Node to node CoreDNS TCP"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 53
  to_port                  = 53
  self                     = true
  security_group_id        = module.eks.node_security_group_id
}

resource "aws_security_group_rule" "_self_coredns_udp" {
  description              = "ARMEN Node to node CoreDNS UDP"
  type                     = "ingress"
  protocol                 = "udp"
  from_port                = 53
  to_port                  = 53
  self                    = true
  security_group_id        = module.eks.node_security_group_id
}

resource "aws_security_group_rule" "_ingress_cluster_control_plane" {
  description              = "CORESTRAP Cluster control plane to node"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 30637
  to_port                  = 31577
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_ingress_cluster_webhook_8443" {
  description              = "CORESTRAP Cluster API to node 8443/tcp webhook"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 8443
  to_port                  = 8443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_ingress_cluster_webhook_9443" {
  description              = "CORESTRAP Cluster API to node 9443/tcp webhook"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 9443
  to_port                  = 9443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_self_ephemeral" {
  description              = "CORESTRAP Node to node ingress on ephemeral ports"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 1025
  to_port                  = 65535
  self                     = true
  security_group_id        = module.eks.node_security_group_id
}

Problem:
When running terraform, I noticed that there were DUPLICATE rules being created (my rules + "default" rules from the eks module)...specifically, this subset of rules was STILL being created by the module, EVEN WHEN node_security_group_enable_recommended_rules=*false*
This totally broke my terraform run, and backing out of it/failing forward took me a while. There's nothing in the documentation that notes "Even if you set this to false (node_security_group_enable_recommended_rules), we're still going to create the following subset of baseline rules..."
This is the specific subset of rules that - no matter what - the eks module will create (even with recommended_rules = false):

resource "aws_security_group_rule" "_ingress_cluster_api" {
  description              = "ARMEN Cluster API to node"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 443
  to_port                  = 443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_egress_cluster_api" {
  description              = "ARMEN Node to Cluster API"
  type                     = "egress"
  protocol                 = "tcp"
  from_port                = 443
  to_port                  = 443
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_ingress_cluster_kubelet" {
  description              = "ARMEN Cluster API to node kubelet"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 10250
  to_port                  = 10250
  security_group_id        = module.eks.node_security_group_id
  source_security_group_id = module.eks.cluster_security_group_id
}

resource "aws_security_group_rule" "_self_coredns_tcp" {
  description              = "ARMEN Node to node CoreDNS TCP"
  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 53
  to_port                  = 53
  self                     = true
  security_group_id        = module.eks.node_security_group_id
}

resource "aws_security_group_rule" "_self_coredns_udp" {
  description              = "ARMEN Node to node CoreDNS UDP"
  type                     = "ingress"
  protocol                 = "udp"
  from_port                = 53
  to_port                  = 53
  self                    = true
  security_group_id        = module.eks.node_security_group_id
}

Describe the solution you'd like.
This can be fixed in one of two ways:

Not great: Just add something in the documentation that says "we're gonna create these, even if this setting is off..."
Great: Add an additional boolean flag that allows:
a.  node_security_group_enable_recommended_rules = true
b.  node_security_group_disable_egress_to_all (or something like it) = true/false

Describe alternatives you've considered.

Additional context

Just to add on this after a day of figuring out why :(
Is that indeed 443 egress from nodes to cluster API is removed from the default node SG rules in v18 it was still present. Seeing that this would normally be needed I think adding this back by default might be a good idea.


Uncaught AttributeError in sendchat.py line 30

Aider version: 0.56.0
Python version: 3.10.9
Platform: Linux-5.15.0-1062-aws-x86_64-with-glibc2.31
Python implementation: CPython
Virtual environment: No
OS: Linux 5.15.0-1062-aws (64bit)
Git version: git version 2.25.1
An uncaught exception occurred:
Traceback (most recent call last):
  File "aider", line 8, in <module>
    sys.exit(main())
  File "main.py", line 698, in main
    return 1
  File "base_coder.py", line 735, in run
    return self.io.get_input(
  File "base_coder.py", line 778, in run_one
    def check_for_urls(self, inp):
  File "base_coder.py", line 1202, in send_message
  File "base_coder.py", line 1885, in auto_commit
    self.io.tool_error(f"Unable to commit: {str(err)}")
  File "repo.py", line 102, in commit
    commit_message = self.get_commit_message(diffs, context)
  File "repo.py", line 187, in get_commit_message
    commit_message = simple_send_with_retries(
  File "sendchat.py", line 38, in wrapper
    retry_exceptions(),
  File "sendchat.py", line 30, in retry_exceptions
    litellm.llms.anthropic.chat.AnthropicError,
AttributeError: module 'litellm.llms.anthropic' has no attribute 'AnthropicError'

Thank you for filing this issue.
The fix is in the latest version (0.57.1), please update aider with
aider --upgrade

Please do not forget to exit and restart aider after it updated itself.
Duplicate #1278


Toxic Rain Pathfinder build added

Your Act X tree image should show the whole tree, even if it's small. It's the last image that people will use for maps as well and should be complete incase they missed something.

Your Act X tree image should show the whole tree, even if it's small. It's the last image that people will use for maps as well and should be complete incase they missed something.

Your final gems only have a 5 link for CA and a 4 link for TR. I would probably have the 6 link by 55 and any Awakened gems by 72. I know most people won't have a 6 link by then, but if they got a tabula it's good to have in the guide.

Your final gems only have a 5 link for CA and a 4 link for TR. I would probably have the 6 link by 55 and any Awakened gems by 72. I know most people won't have a 6 link by then, but if they got a tabula it's good to have in the guide.

Added complete passive tree with 120 points spent.
I still haven't added a 6-links or Awakened Gems because from my point of view, after finishing Acts people will ned to decide which complete high-end guide to follow, so it doesn't make a lot of sense to add high-level skill sets since they can be very different for people.

Added complete passive tree with 120 points spent.
I still haven't added a 6-links or Awakened Gems because from my point of view, after finishing Acts people will ned to decide which complete high-end guide to follow, so it doesn't make a lot of sense to add high-level skill sets since they can be very different for people.

Fixed all issues found by reviewer.

Fixed all issues found by reviewer.


Hashed chunk ids

Feature request

What is the expected behavior?
Hashed chunk ids
What is motivation or use case for adding/changing the behavior?
After this PR we have got better long-term caching "out of the box", but we have many chunks and they have numeric ids and these ids changes very often and it still affects long-term caching.
Named chunk ids is too large
How should this be implemented in your opinion?
Clone HashedModuleIdsPlugin with the same login and adopt it for chunks. It looks like an easy solution...
Are you willing to work on this yourself?
yes

There is a open card for this in the webpack 5 project.
Better skip the hashed chunk ids and use the deterministic chunk ids algorithm similar to modules.
deterministic chunk ids is based on named chunk ids + the hashing part.

Are you about clone the DeterministicModuleIds and adopt it for chunks?

Currently not, but this is planned. But you can go ahead and send a PR.

👌


Project name and trademark usage

Hi Isaac,
It's cool to see people making automation tools for VyOS. However, the name "VyOS" is a registered trademark in the US (where you are located) and other countries, and we cannot allow its use without our permission, not least because we'll risk losing it and making it vulnerable to bad-faith actors if we do.
Just like if you made a tool for, say, RedHat Enterprise Linux, you could not name it "RedHat Enterprise Linux Firewall GUI" — that would misrepresent the tool as if it was produced or affiliated with RedHat, and any possible issues with it would harm the reputation of RedHat. Just like RedHat, we cannot allow that.
You should change the project name to make it clear that it's not affiliated with VyOS. If you don't, we can report it to GitHub for a repository takedown or take a legal action, but we hope that we can cooperate and it will not get to that.
P.S.
VyOS has HTTP APIs now that are far more reliable than SSH connections, you should consider using them.

Thank you for reaching out @dmbaturin .  I appreciate what you are saying and I was not looking infringe on the VyOS name or trademark.  I do want to make the tool findable for users of VyOS who would find it useful, but I fully respect and appreciate the great work that the developers of VyOS do and the product you are making available to the community.
What would you see as an appropriate name or way to reference VyOS?  Is there a document that describes the ways that the community can reference your brand?
Would the following be acceptable?
Firewall GUI for use with VyOS
I am curious what drew your attention today vs the last 6 weeks and my posting about the work I was doing in the VyOS forum on 1/15/2024.  Did I do something or post something in a more egregious way?
Again, I appreciate what you are saying and want to make changes to fully alleviate your concern.  I want add to the ecosystem and not detract from it.
Thanks!

There's nothing egregious, it was just in my backlog.
Regarding findability: I think it's ok to write something like "${name} is a firewall management tool for VyOS". We need to develop trademark guidelines for sure — it's a good reminder to work on them.
We should also add a website section for third-party tools and resources and your project would be a good fit for it.

Thanks for the reply @dmbaturin.  Given the changes made in #21, is it okay to close this issue?

Yes, the branding looks good to me now, thanks!


install error，I have uninstall lightgbm

INFO:LightGBM:Starting to compile the library.
INFO:LightGBM:Starting to compile with CMake.
WARNING:LightGBM:Compilation failed.
INFO:LightGBM:Starting to compile with Homebrew OpenMP paths guesses.
WARNING:LightGBM:Compilation failed.
INFO:LightGBM:Starting to compile with MacPorts OpenMP paths guesses.
Traceback (most recent call last):
File "setup.py", line 76, in silent_call
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['make', '_lightgbm', '-j4']' returned non-zero exit status 2.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 310, in
setup(name='lightgbmmt',
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/Users/zouyapeng/opt/anaconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "setup.py", line 267, in run
compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_mpi=self.mpi,
File "setup.py", line 198, in compile_cpp
silent_call(["make", "_lightgbm", "-j4"], raise_error=True,
File "setup.py", line 85, in silent_call
raise Exception("\n".join((error_msg, LOG_NOTICE)))
Exception: An error has occurred while building lightgbm library file
The full version of error log was saved into /Users/zouyapeng/LightGBM_compilation.log

It seems you have a problem when compiling the cpp. You may fix the problem by checking the log /Users/zouyapeng/LightGBM_compilation.log.
Or you can send me the log here.


rename ci scripts to match the branch

I have been wanting to change<EMAIL_ADDRESS>to DerYeger/yarn-setup-action@master in previous PR.

I have been wanting to change<EMAIL_ADDRESS>to DerYeger/yarn-setup-action@master in previous PR.

It's risky to point to the master branch of an action in case it makes changes that break your CI. I would prefer we stick to a specific version that we know works.

May I ask why not "v3.19.0"?

May I ask why not "v3.19.0"?

The other platforms use 3.18.3. It's best to use the same version for all the tests.  I'm hoping parts of the setup could be made common across platforms at some point.


Steam Overlay fails to hook into Device Context Creation and SwapBuffers

I am attempting to integrate the Steam Overlay with my Windows game, however it doesn't render on top of the window.
I created a new blank test project and used another OpenGL C# wrapper, and the Steam Overlay worked successfully. I then switched this test project over to OpenGL.Net, however the Steam Overlay was invisible again.
The Steam Overlay works by hooking into the device creation, as well as hooking into the swap buffers call and doing its rendering there. The Steam documentation specifies that the overlay must be initialised 'prior to initializing the OpenGL/D3D device, otherwise it won't be able to hook the device creation'.
Is there anything happening in the way OpenGL.Net creates the device context that could be confusing the Steam Overlay, or causing it to hook into the wrong context?

Maybe there is a conflict with the static constructor of Gl. Probably Steam hooking is based on DLL injection, wrapping well known libraries (such as opengl32.dll on Windows). But the Gl static constructor forces the loading of the libraries by creating a temporary GL context. You could try to disable the Initialize call in static constructor by using user environment variables prior to call any OpenGL.Net method.
Or.. are you running with EGL? Maybe EGL device context creation is not supported by Steam (I doubt it)...


Min with DST timeslot labels are incorrect

Do you want to request a feature or report a bug?
Bug
What's the current behavior?
When displaying events on calendar with min and max props set, the event is displayed in the correct spot, but the hour on the timeslot label is off by 1 hour (assuming this is DST related).
CodeSandbox

What's the expected behavior?
I would expect the timeslot labels to have the correct time label based on whether you are currently in or out of Daylight Savings Time (DST).   I am looking through the source code to see if I can find the exact error.

It seems like max has been fixed in version 0.20.2 on npm. min still not working correctly for me.

Thank you @NuclearKev, I checked and it does work if I don't have min set.   I have updated the issue title since max is resolved.

I noticed this appears to only be an issue for the week when DST changed.  I am assuming it has something to do with the extra hour on Sunday for the week view.  I haven't been able to find a solution yet.

I came across this problem with the 'min' prop set, and found that by localizing my calendar to have the week start on Monday, then the problem "went away" (I'm sure it's still there, but just not noticeable). Hopefully this helps identify where the problem lies!
Before:
const localizer = BigCalendar.momentLocalizer(moment);
After:
moment.locale("au", {
week: {
dow: 1,
doy: 1
}
});
const localizer = BigCalendar.momentLocalizer(moment);
My test period was the week that included Sunday 7th April 2019 in Australian timezone (which is the end of daylight savings), and I have a 'min' set like this:
const CALENDAR_MIN = new Date(1970, 6, 1, 6);
...
<DragAndDropCalendar
...
min={CALENDAR_MIN}
...
When the week started on Sunday, the problem was exactly as per screenshots. When the week started on Monday, then all was visibly OK.


How might we obtain X,Y location of the detections?

Hi @Yiming-M , thank you for sharing the great work youve done with crowd counting.
I am looking through the test_nwpu.py file and was thinking how might we obtain the X,Y coordinates of each detected head?

For example, in your visualisation, we could see the dots that matches each person's head. Is there a way we can get these X and Y positions?

@Yiming-M Sorry for the late reply. I see, so i understand it is a probability map and density map.
Would it be possible to apply something like a threshold such as in this implementation of another research paper that is also related to crowd counting? (https://github.com/taohan10200/IIM) title Independent instance maps
Where a threshold and mask is applied to the prediction map to get the individual heads?


Unable to train

First of all thank you for your amazing work. I have a question about training. I am trying to train MeanScaleHyperprior network with (192, 192) channels using vimeo90k set. I randomly crop 256*256 patches, use 16 as batch size and set lamda=0.0018 (quality 1). However I am not able to properly train the network so that the results match with the plots on https://interdigitalinc.github.io/CompressAI/zoo.html. There is one small difference on reading images. I use imageio.imread which returns 0-255 range, then I normalize by dividing 255. I observe no significant change after 500k iterations and stuck at 26.86 dB, 0.30 bpp on Kodak dataset. Do you have any guess what may be the reason? Thanks

Hi! yes this does not look right. are you using the example training script? what's your learning rate?
If you can provide you training code this could be helpful also!
(I don't think this should be related to using imageio)

I am using the following code for training.
# coding: utf-8

# In[1]:

import torch
import torch.nn as nn
from torch.autograd import Variable
import numpy as np
import glob
import os
import itertools
import warnings
warnings.filterwarnings('ignore')
import imageio
from natsort import natsorted
from skimage.measure import compare_ssim,compare_psnr,compare_mse
import logging
import time
import random
import torch.nn.functional as F

from compressai.models import MeanScaleHyperprior
from rd import RateDistortionLoss

device = torch.device("cuda:0")

logging.basicConfig(filename="compression.log",level=logging.INFO)

def normalize(tensor):
    return tensor/255.

def float_to_uint8(image):
    clip = np.clip(image*255,0,255)
    im_uint8 = np.round(clip).astype(np.uint8).transpose(1,2,0)
    return im_uint8

def MSE(gt,pred):
    mse = np.mean((gt-pred)**2)
    return mse

def PSNR(mse,data_range):
    psnr = 10*np.log10((data_range**2)/mse)
    return psnr

def calculate_distortion_loss(out,real):
    distortion_loss = torch.mean((out-real)**2)
    return distortion_loss

def calculate_rate_loss(likelihood):
    return -torch.sum(torch.log2(likelihood))

def train_image_loader(path="/home/ml/Akin/vimeo_septuplet/sequences/"):
    images = []
    fold = natsorted(glob.glob(path+"*"))
    for f in fold:
        mfold = natsorted(glob.glob(f+"/*"))
        for m in mfold:
            imgs = natsorted(glob.glob(m+"/*.png"))
            for i in imgs:
                images.append(i)
    return images

def prepare_train_data(image_batch):

    X_train = []

    patch_size = 256

    for ims in image_batch:

        img_orig = imageio.imread(ims)

        x = random.randint(0, img_orig.shape[1] - patch_size)
        y = random.randint(0, img_orig.shape[0] - patch_size)
        img = img_orig[y:y+patch_size, x:x+patch_size].transpose(2,0,1)

        X_train.append(img)

    X_train = np.array(X_train)
    Y_train = X_train.copy()

    return X_train,Y_train

def test_image_loader(path="/home/ml/Akin/Kodak"):
    images = natsorted(glob.glob(path+"/*.png"))

    return images

def prepare_test_data(im):
    X_test = []

    img = imageio.imread(im).transpose(2,0,1)

    X_test.append(img)

    X_test = np.array(X_test)
    Y_test = X_test.copy()

    return X_test,Y_test

def train_one_step(im_batch,model,optimizer,criterion,aux_optimizer):

    model = model.train()

    X_train,Y_train = prepare_train_data(im_batch)
    X_train = normalize(torch.from_numpy(X_train).to(device).float())
    Y_train = normalize(torch.from_numpy(Y_train).to(device).float())

    output = model.forward(X_train)

    loss_dict = criterion(output, Y_train)

    loss = loss_dict["loss"]
    dist_loss = loss_dict["mse_loss"]
    rate_loss = loss_dict["bpp_loss"]

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    aux_optimizer.zero_grad()
    aux_loss = model.aux_loss()
    aux_loss.backward()
    aux_optimizer.step()

    return dist_loss.item(),rate_loss.item(),loss.item(),aux_loss.item()

def validate(model,criterion):

    with torch.no_grad():

        average_distortion = 0

        average_mse = 0

        total_pixels = 0

        average_test_loss = 0
        average_bpp = 0

        model = model.eval()

        test_images = test_image_loader()

        total_images = len(test_images)

        d = 0
        os.makedirs("Created/",exist_ok=True)

        for ims in test_images:
            X_test,Y_test = prepare_test_data(ims)

            (m,c,w,h) = X_test.shape
            total_pixels += m*w*h

            X_test_ = normalize(torch.from_numpy(X_test).to(device).float())
            Y_test_ = normalize(torch.from_numpy(Y_test).to(device).float())

            output = model.forward(X_test_)
            out = output["x_hat"]

            loss_dict = criterion(output, Y_test_)
            loss = loss_dict["loss"]
            dist_loss = loss_dict["mse_loss"]
            rate_loss = loss_dict["bpp_loss"]

            average_test_loss += loss.item()
            average_bpp += rate_loss.item()

            uint8_out = float_to_uint8(out[0,:].cpu().detach().numpy())
            uint8_real = float_to_uint8(Y_test_[0,:].cpu().numpy())

            o_f = uint8_out.astype(np.float32)/255.
            r_f = uint8_real.astype(np.float32)/255.

            mse_dec = MSE(r_f,o_f)

            average_mse += mse_dec

            save_path = "Created/"+"img"+str(d)+".png"

            imageio.imsave(save_path,uint8_out)

            d += 1

    average_mse /= total_images

    average_psnr = PSNR(average_mse,data_range=1)

    average_bpp /= total_images

    average_test_loss /= total_images

    return average_test_loss,average_psnr,average_bpp

def save_model(model,optimizer,name):
    state = {
        "state_dict" : model.state_dict(),
        "optimizer" : optimizer.state_dict()
    }
    torch.save(state,name+".pth")

def main():
    np.random.seed(1)
    torch.manual_seed(2)
    random.seed(3)

    total_train_step = 2000000
    train_step = 1000
    lr_step = 1000000
    learning_rate = 5.e-5
    model = MeanScaleHyperprior(192,192)
    model = model.to(device).float()

    criterion = RateDistortionLoss(lmbda=0.0018).to(device).float()

    optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=learning_rate)
    aux_optimizer = torch.optim.Adam(model.aux_parameters(), lr=5.e-5)

    model_parameters = filter(lambda p: p.requires_grad, model.parameters())
    params = sum([np.prod(p.size()) for p in model_parameters])
    logging.info("number of compress parameters: "+str(params))

    train_images = train_image_loader()

    batch_size = 16

    step_train_loss = 0
    step_distortion_loss = 0
    step_rate_loss = 0
    step_aux_loss = 0

    best_test_loss = 10**10

    time_start = time.time()

    for minibatch_processed in range(1,total_train_step+1):

        image_batch = random.sample(train_images, batch_size)

        total_distortion_loss,total_rate_loss,loss,aux_loss = train_one_step(image_batch,model,optimizer,criterion,aux_optimizer)

        step_train_loss += loss
        step_distortion_loss += total_distortion_loss
        step_rate_loss += total_rate_loss
        step_aux_loss += aux_loss

        if minibatch_processed % train_step == 0:
            average_test_loss,average_psnr_dec,average_bpp = validate(model,criterion)
            if average_test_loss < best_test_loss:
                save_model(model,optimizer,"compression")
                best_test_loss = average_test_loss
                psnr_dec_at_best_loss = average_psnr_dec
                bpp_at_best_loss = average_bpp
                logging.info("NEW BEST !!!")

            logging.info("learning rate: "+str(learning_rate))
            logging.info("iterations: "+str(minibatch_processed))
            logging.info("distortion loss: "+str(step_distortion_loss/train_step))
            logging.info("rate loss: "+str(step_rate_loss/train_step))
            logging.info("train loss: "+str(step_train_loss/train_step))
            logging.info("aux loss: "+str(step_aux_loss/train_step))
            logging.info("--------")
            logging.info("test psnr dec: "+str(average_psnr_dec))
            logging.info("test bpp: "+str(average_bpp))
            logging.info("test loss: "+str(average_test_loss))
            logging.info("best test loss: "+str(best_test_loss))
            logging.info("^^^")
            logging.info("psnr dec at best test loss: "+str(psnr_dec_at_best_loss))
            logging.info("bpp at best test loss: "+str(bpp_at_best_loss))
            logging.info("**************")

            step_train_loss = 0
            step_distortion_loss = 0
            step_rate_loss = 0
            step_aux_loss = 0

        if minibatch_processed % lr_step == 0:
            learning_rate = 1.e-5
            for g in optimizer.param_groups:
                g["lr"] = learning_rate

if __name__=='__main__':
    main()

I first set learning rate to 1e-4 but my training loss blows up somewhere. So I set it to 5e-5.

to solve the stability issues you can use gradient clippping on the norm (like here https://github.com/InterDigitalInc/CompressAI/blob/master/examples/train.py#L123). We've had good results with torch.nn.utils.clip_grad_norm_(model.parameters(), 1)
I was going to suggest to set the learning rate to 1e-5 after enough iterations, but looks like your code is already doing this.
if you are training the models from scratch I would recommend using a stronger learning rate for the auxiliary optimizer (like 1e-3)

Thank you Jean. One last question. Do you apply downsampling to vimeo90k set to reduce compression artefacts? Is it also enough to use random cropping only or do you apply flipping, rotation etc ?

We simply extract random 256x256 cropped patches from the vimeo90k triplet dataset (http://toflow.csail.mit.edu/index.html#triplet). The triplet frames were already resized to 448x256 by the dataset authors. We don't apply any flipping or rotation.

Thank you very much for your quick reply and this amazing work again.

Thanks! I'm closing this for now. Feel free to re-open an issue if you still encounter issues during training.


Unable to build in Windows - GEOBLAZE_WEBPACK_ENTRY is not a recognozed command

In vanilla fork, did npm install, and npm run build. That led to error saying something like:
GEOBLAZE_WEBPACK_ENTRY is not a recognozed command
Apparently setting environment variables works differently on Windows and Linux, and this project appears to have been coded on/for Linux.

With above two commits, issue resolved. Closing.


Use syntax highlighting in README and simplified.

Showing the different units right in the code is a lot easier to understand.
Great library, thanks for creating!

Thank you

:+1:
You bet. Thanks for writing this. Does exactly what you'd expect it to. Very clear and concise.


Add support for unit testing - jest

Hey @sheshbabu , I'd like to help with adding unit testing

@gibsonhan Awesome, thanks for the interest! 👍
Can you create a PR with jest config for both client and server sections and 1-2 example unit tests - maybe NavBar.tsx in client [1] and DateTimeUtil.ts in server [2].
Let me know if you prefer to approach this in a different way 😊
[1] https://github.com/sheshbabu/freshlytics/blob/master/src/client/components/Navbar.tsx
[2] https://github.com/sheshbabu/freshlytics/blob/master/src/server/utils/DateTimeUtil.ts

I am new to docker. I managed to get the build running. But I am unsure what the "public url" is

But I am unsure what the "public url" is

It's the URL your use to access the server. If you're running in local, it'd be http://localhost:3001


Roman rubin final

Updates for latest throughputs for lsst and roman and config file changes for v1.0.1

This PR introduces a new dependency on galsim, which is used to retrieve roman filters. A new test of this new feature should be included before this PR is merged into main.


Listen for widevine events + update docs

As documented in #1153, for Widevine support to work properly, we need to listen for the Widevine ready event, and as well for certain sites the app must be signed.
This PR adds the events, and as well adds better documentation on the signing limitation.
This may also help resolve #1147

@TheCleric I merge your doc with mine added minutes ago before I saw this PR.


ci: add action for unit test

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@76b3ff5). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #1   +/-   ##
=======================================
  Coverage        ?   60.00%
=======================================
  Files           ?        4
  Lines           ?      285
  Branches        ?        0
=======================================
  Hits            ?      171
  Misses          ?      108
  Partials        ?        6

Flag
Coverage Δ

unittests
60.00% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76b3ff5...dd1d70f. Read the comment docs.


Year bug

Includes unit handling that converts from hours to years.
Also, replaces tabs with 4 spaces.

Hello @katyhuff! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file offsets/offsets.py:

Line 91:69: W291 trailing whitespace
Line 100:56: W291 trailing whitespace
Line 103:17: W291 trailing whitespace
Line 104:75: W291 trailing whitespace
Line 106:48: W291 trailing whitespace
Line 108:71: W291 trailing whitespace
Line 110:66: W291 trailing whitespace
Line 111:53: W291 trailing whitespace
Line 115:1: W293 blank line contains whitespace
Line 116:13: W291 trailing whitespace
Line 123:5: E122 continuation line missing indentation or outdented
Line 123:18: E231 missing whitespace after ':'
Line 123:27: E231 missing whitespace after ','
Line 124:5: E122 continuation line missing indentation or outdented
Line 124:12: E231 missing whitespace after ':'
Line 134:5: E122 continuation line missing indentation or outdented
Line 134:45: W291 trailing whitespace
Line 135:5: E122 continuation line missing indentation or outdented
Line 135:12: E231 missing whitespace after ':'
Line 136:5: E122 continuation line missing indentation or outdented
Line 142:5: E122 continuation line missing indentation or outdented
Line 143:5: E122 continuation line missing indentation or outdented
Line 144:5: E122 continuation line missing indentation or outdented
Line 145:5: E122 continuation line missing indentation or outdented

My understanding is that the variable lifetime assumed [years] and it should be in [hours] because the costs are give as [$/unit-hours].

Yes. The variable itself is easiest to provide in years, though, so that part is right. Thanks!


Mini.completion as more of a tab complete

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md

Module(s)
completion
Description
I have been really enjoying using mini.completion, and have enjoyed that it is only sourcing suggestions from text previously written.
Apologies if this is already possible with the config.
A nice improvement would be to auto use the first suggestion after pressing tab.

I am not entirely sure what you mean here.
There are suggested mappings for <Tab> and <S-Tab> in help file. With them and default config.set_vim_settings (which sets completeopt option), you can press <Tab> while popup suggestions are visible which will lead to first suggestion placed in text. After that, you can continue typing text as usual. This is how I use it.
If you also want for dismiss popup after <Tab>, try this mapping (adds <C-y> in case of visible popup):
vim.api.nvim_set_keymap('i', '<Tab>', [[pumvisible() ? "\<C-n>\<C-y>" : "\<Tab>"]], { noremap = true, expr = true })

... or a more modern approach:
vim.keymap.set('i', '<Tab>', [[pumvisible() ? "\<C-n>\<C-y>" : "\<Tab>"]], { expr = true })

Does this solve your issue?


[Bug]: Ghost on a sub directory no longer works?

Description
Hi, some months ago I was able to put Ghost on a subdirectory of another website in Coolify
Let's say I have

Website A (Laravel)
Website B (Ghost)

I could have B inside A on a subdirectory like example/blog
But now it seems like requests are not redirected to website B, it shows 404 of website A
Minimal Reproduction (if possible, example repository)

Create a website of any kind
Create a ghost website
Set the URL to something like https://example.com:2368/blog
It does not work :(

Exception or Error
No response
Version
v4.0.0-beta.323
Cloud?

[ ] Yes
[X] No

Basically this commit allowed ghost on subdirector
https://github.com/coollabsio/coolify/commit/543f983e41f594483d54fb2f769234ebf1e2863d

Basically this commit allowed ghost on subdirectory @andrasbacsai
https://github.com/coollabsio/coolify/commit/543f983e41f594483d54fb2f769234ebf1e2863d

Okay, the issue is having multiple ghost services, the traefik.http.middlewares.redir-ghost middleware is clashing with other projects, because it's duplicated
Please fix it @andrasbacsai by making them unique based on the project name

I added a fix in v364 please try again now and let me know if it works.

It works, thanks a lot <3


Add relatedChart field in chart config to indicate whether related chart can be scaled

i wonder if we should instead explicitly define a denominator (and if not defined, do not scale the chart). we often scale by population, when we should scale by a subpopulation.

Added a few denominator cases, otherwise default to total population

ptal with the new change


bugfix: judge enablement of libnetwork resolver by network mode

Signed-off-by: mathspanda<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
The PR is to judge whether enabling embedded dns server by network mode.
If network mode is user-defined, container uses embedded dns server.
If not, container uses resolv.conf file on host directly.
Ⅱ. Does this pull request fix one issue?

Issue #2362
Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
None
Ⅳ. Describe how to verify it
# content of resolv.conf file is the same with one on host
pouch run busybox cat /etc/resolv.conf

# content of resolv.conf file contains "nameserver <IP_ADDRESS>"
pouch run --net ns1 busybox cat /etc/resolv.conf

Ⅴ. Special notes for reviews
None

@rudyfly PTAL.

Could you add me as a friend on wechat for further discussion. my account is shlallen. @mathspanda

@mathspanda Thank you for your contributions, LGTM


Non UVA Users

non-uva users (folks from the community ... i.e. don't have UVa computing id / can't Netbadge in) should be able to create their own account
When setting up this flow, be certain that UVA folks will continue to use Shibboleth to create their account and log in.
Possibly add the ability to use another Open Auth provider, such as Facebook/Google/Amazon.
Requested by: @vpnagraj

Dan, if we move forward with this, I have a couple of ideas. We might also integrate Oauth into the SSO tool I wrote for Discourse -- and just expand the providers instead of only Netbadge.


Update links.json

Added projects presented in Feb PG WG Meetings

Thank you for your contribution. I see gashawk but don't seem to see other logos though
~/.../ens/ensdomains-v2 (master)$ls src/components/Home/Ecosystem/*/gashawk*
src/components/Home/Ecosystem/apps/gashawk.jpeg

~/.../ens/ensdomains-v2 (master)$ls src/components/Home/Ecosystem/*/safetransfer*
ls: src/components/Home/Ecosystem/*/safetransfer*: No such file or directory

~/.../ens/ensdomains-v2 (master)$ls src/components/Home/Ecosystem/*/1w3*
ls: src/components/Home/Ecosystem/*/1w3*: No such file or directory


tags incorrectly wrapped in  tags

I am parsing markdown containing html and have the following setup:
const remark = new Remark().data(`settings`, {
  commonmark: true,
  footnotes: true,
  pedantic: true,
})

If I parse an iframe tag, I get the expected result:
const markdownAST = remark.parse(`

{
	"type": "root",
	"children": [{
		"type": "html",
		"value": "",
			"position": {
				"start": {
					"line": 2,
					"column": 1,
					"offset": 1
				},
				"end": {
					"line": 3,
					"column": 5,
					"offset": 79
				},
				"indent": [1]
			}
	}],
	"position": {
		"start": {
			"line": 1,
			"column": 1,
			"offset": 0
		},
		"end": {
			"line": 3,
			"column": 5,
			"offset": 79
		}
	}
}

However if I parse an object tag:
const markdownAST = remark.parse(`
<object url="http://www.example.com/" width="100%" height="400"></object>

The result is wrapped in paragraph tags.
 {
 	"type": "root",
 	"children": [{
 		"type": "paragraph",
 		"children": [{
 			"type": "html",
 			"value": "<object url="http://www.example.com/" width="100%" height="400">",
 			"position": {
 				"start": {
 					"line": 2,
 					"column": 1,
 					"offset": 1
 				},
 				"end": {
 					"line": 2,
 					"column": 65,
 					"offset": 65
 				},
 				"indent": []
 			}
 		}, {
 			"type": "html",
 			"value": "</object>",
 			"position": {
 				"start": {
 					"line": 2,
 					"column": 65,
 					"offset": 65
 				},
 				"end": {
 					"line": 2,
 					"column": 74,
 					"offset": 74
 				},
 				"indent": []
 			}
 		}, {
 			"type": "text",
 			"value": "\n    ",
 			"position": {
 				"start": {
 					"line": 2,
 					"column": 74,
 					"offset": 74
 				},
 				"end": {
 					"line": 3,
 					"column": 5,
 					"offset": 79
 				},
 				"indent": [1]
 			}
 		}],
 		"position": {
 			"start": {
 				"line": 2,
 				"column": 1,
 				"offset": 1
 			},
 			"end": {
 				"line": 3,
 				"column": 5,
 				"offset": 79
 			},
 			"indent": [1]
 		}
 	}],
 	"position": {
 		"start": {
 			"line": 1,
 			"column": 1,
 			"offset": 0
 		},
 		"end": {
 			"line": 3,
 			"column": 5,
 			"offset": 79
 		}
 	}
 }

[Update] I hadn't realised an object tag wasn't a block element, so perhaps this is just how you hand things, but I couldn't find any mention of wrapping inline elements in paragraph tags. If this is the expected behaviour please could you point me towards the docs.
I'm using Remark 7.1

hey hey, sorry for the slow reply!
This is expected behaviour, as you noted in your edit: <object> isn’t "block". You can see the same thing happen on commonmark too!
I think I can close this now, but feel free to comment below if you disagree and I’ll reopen this issue.

No problem. But was a bit of a surprise. Is it documented anywhere?