Multi-parameter gradient information from PyBOP to Pints

@martinjrobins would you mind taking a look at the implementation of the sensitivities in simulateS1() and the example gradient_descent.py? I've found that this will work for single parameter, but completely breaks down for multi-parameter. Pints is expecting an array of shape (n_times, n_parameters), which I believe is sorted (although I'm not sure we need to convert from casadi.DM to numpy..
For more than a single parameter, the parameter estimation wanders throughout the space without converging. I suspect the gradient information isn't being correctly passed to Pints. The code is very brittle and hacked together at the moment as I'm working through understanding the interfaces (i.e. not to be merged).

Closing this — solution found. Gradient information wasn't passed correctly when using the reshape() method. Returning the transposed gradient array to pints resulted in correct gradient based optimization.


hidden doesn't work

I have two files
test.js :
require('./test2')()
console.log('hello')

and test2.js :
module.exports = function(){console.log('hello')}

To my understanding, when I type
node-debug test.js --hidden test2

it should not show any of the code excuted in test2.js in the debugger, but it does.
Am I missing something?

Please submit that this issue fixed in 0.9.0

Closing this issue as fixed, feel free to reopen if you can reproduce the problem using the latest version of Node Inspector.

Can you reference the commit that resolved this? I am still seeing issues using the rc file.

@3y3, @bajtos, I can confirm this issue exists using 0.9.2.  Do you want to reopen this issue, or should I create a new issue?
I am currently using node v0.10.32.  I created test.js and test2.js as below for an easier test:
test.js
var test2 = require('./test2');
console.log('hello 1');
test2();

test2.js
module.exports = function() {
  console.log('hello 2');
}

Then I run node-debug test.js --hidden 'test2'.  The debugger loads with script execution paused, I switch to test2.js, add a breakpoint on the console.log line, resume execution, and the debugger breaks on my breakpoint in test2.js, even though that file should be ignored because of --hidden.

@zacronos Can you reproduce the problem on 0.7.x version too? What is your platform?

@bajtos, I see the same behavior on 0.7.4 of node-inspector.
I am using node v0.10.32 on OS X 10.9.5 with Chrome 41.0.2272.89 (64-bit).  If there is any other information you'd like or scenarios you'd like me to test, let me know.
Thanks for the fast response!

@zacronos , you start debugger with wrong command line:
your - node-debug test.js --hidden 'test2'
expected - node-debug --hidden 'test2' test.js
With correct commant line I can't reproduce your issue.

@zwhitchcox , same as @zacronos
I close this issue. You feel free to reopen it if I'm wrong here.


gmssl命令行如何生成私钥？

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

gmssl命令行如何生成私钥？

@herobigdata2099  gmssl sm2 -genkey -out skey.pem


Make scour not to close the output file?

How do you make the API not to close the output file? In particular, my code is the following:
with File(open(tempPrimitive, 'rb')) as notMinimized, File(open(".temp2.svg", 'wb+')) as minimized:
            start(None, notMinimized, minimized) #Minimize the svg art
            photo.placeholder.save('ph.svg', minimized) #ValueError: seek of closed

I believe that an option could be used on the options class (first argument on the start method call, currently set to None). Additionally, any documentation on this options class would be really helpful.

As start is basically the entry point for using scour itself from the command line, there's no way to avoid closing the file using this function.
I'd currently recommend to use scourString instead, so you can handle reading and writing however you like.
Long-term we'd like to implement a proper public API (see #119), that will most likely also handle file-like objects then.


Error: Exited with 4, Something wrong with the build process, Error: Command "vercel build" exited with 1

Bug report

<EMAIL_ADDRESS>and<EMAIL_ADDRESS>both tested.
URL:  https://xbot-febc-git-preview-rly.vercel.app/
Repository:  https://github.com/guoxiangke/xbot-febc/commit/4e674fd1567bb7d0dca3e2d2e244470f681b3666

Description
Hello, I have a laravel project which works find if I remove 3 packages in the composer.json
but when I require one, the build give errors like below.
thanks.

[21:21:55.338] Running build in Washington, D.C., USA (East) – iad1
[21:21:55.459] Cloning github.com/guoxiangke/xbot-febc (Branch: preview, Commit: 4e674fd)
[21:21:55.735] Cloning completed: 276.026ms
[21:21:58.191] Restored build cache
[21:21:58.285] Running "vercel build"
[21:21:58.802] Vercel CLI 33.5.1
[21:21:59.768] > Installing Builder<EMAIL_ADDRESS>[21:22:02.890] Installing dependencies...
[21:22:03.645] yarn install v1.22.17
[21:22:03.679] [1/4] Resolving packages...
[21:22:03.718] success Already up-to-date.
[21:22:03.723] Done in 0.08s.
[21:22:03.737] Running "yarn run build"
[21:22:03.949] yarn run v1.22.17
[21:22:03.977] $ vite build
[21:22:04.246] vite v5.1.2 building for production...
[21:22:04.286] transforming...
[21:22:04.541] ✓ 47 modules transformed.
[21:22:04.581] rendering chunks...
[21:22:04.597] computing gzip size...
[21:22:04.601] public/build/manifest.json            0.25 kB │ gzip:  0.13 kB
[21:22:04.601] public/build/assets/app-l0sNRNKZ.js   0.00 kB │ gzip:  0.02 kB
[21:22:04.602] public/build/assets/app-CifqVuM1.js  29.83 kB │ gzip: 11.98 kB
[21:22:04.602] ✓ built in 328ms
[21:22:04.612] Done in 0.67s.
[21:22:04.638] 🐘 Downloading user files
[21:22:04.638] 🐘 Downloading PHP runtime files
[21:22:04.642] 🐘 Installing Composer dependencies [START]
[21:22:04.684]
[21:22:04.685] Warning: PHP Startup: Unable to load dynamic library 'mongodb' (tried: /vercel/path0/.vercel/builders/node_modules/@libphp/amazon-linux-2-v81/native/php/modules/mongodb (/vercel/path0/.vercel/builders/node_modules/@libphp/amazon-linux-2-v81/native/php/modules/mongodb: cannot open shared object file: No such file or directory<EMAIL_ADDRESS>(libutf8proc.so.1: cannot open shared object file: No such file or directory)) in Unknown on line 0
[21:22:04.685]
[21:22:04.686] Warning: PHP Startup: Unable to load dynamic library 'swoole' (tried: /vercel/path0/.vercel/builders/node_modules/@libphp/amazon-linux-2-v81/native/php/modules/swoole (/vercel/path0/.vercel/builders/node_modules/@libphp/amazon-linux-2-v81/native/php/modules/swoole: cannot open shared object file: No such file or directory<EMAIL_ADDRESS>(libodbc.so.2: cannot open shared object file: No such file or directory)) in Unknown on line 0
[21:22:04.921] [7.4MiB/0.11s] Installing dependencies from lock file
[21:22:04.924] [7.6MiB/0.11s] Verifying lock file contents can be installed on current platform.
[21:22:04.932] [7.8MiB/0.12s] Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. It is recommended that you run `composer update` or `composer update <package name>`.
[21:22:04.935] [8.1MiB/0.12s] - Required package "madcoda/php-youtube-api" is not present in the lock file.
[21:22:04.935] [8.1MiB/0.12s] - Required package "metowolf/meting" is not present in the lock file.
[21:22:04.936] [8.1MiB/0.12s] - Required package "voku/simple_html_dom" is not present in the lock file.
[21:22:04.936] [8.1MiB/0.12s] This usually happens when composer files are incorrectly merged or the composer.json file is manually edited.
[21:22:04.936] [8.1MiB/0.12s] Read more about correctly resolving merge conflicts https://getcomposer.org/doc/articles/resolving-merge-conflicts.md
[21:22:04.936] [8.1MiB/0.12s] and prefer using the "require" command over editing the composer.json file directly https://getcomposer.org/doc/03-cli.md#require-r
[21:22:04.936] [8.1MiB/0.12s] Memory usage: 8.09MiB (peak: 9.43MiB), time: 0.12s
[21:22:04.950] Error: Exited with 4
[21:22:04.950]     at ChildProcess.<anonymous> (/vercel/path0/.vercel/builders/node_modules/vercel-php/dist/utils.js:182:24)
[21:22:04.950]     at ChildProcess.emit (node:events:517:28)
[21:22:04.950]     at ChildProcess.emit (node:domain:489:12)
[21:22:04.950]     at ChildProcess._handle.onexit (node:internal/child_process:292:12)
[21:22:04.981] Error: Command "vercel build" exited with 1
[21:22:05.045] Command "vercel build" exited with 1
[21:22:05.122]

I test with a new project, it works when init deploy.
but when continue build and deploy, it fails.

need redeploy without cache, now build with no error.
But it should not redeploy each time !

Check my comment here.

Can you test latest version 0.7.3 (at this moment)?


Add support for ASG max instance lifetime

PR o'clock
Description
Adds support for the max instance lifetime parameter in the ASG. This will allow old instances (those running longer than a specified number of seconds) to be automatically terminated.
Fixes #768.
Checklist

[x] Change added to CHANGELOG.md. All changes must be added and breaking changes and highlighted
[x] CI tests are passing
[x] README.md has been updated after any changes to variables and outputs. See https://github.com/terraform-aws-modules/terraform-aws-eks/#doc-generation

Nice. Thanks @sidprak


RFC: introduce the STM32 system wakeup pins to exit low power modes

Introduction
The stm32 mcu has several external wakeup pins (GPIO) that are able to trigger a wakeup from Standby or Shutdown low power modes.
The WakeUp pins are fixed GPIO pins.
Each WakeUp pin is programmed previous to entering a low power standby/shutdown and will exit the system on edge-detection.
The stm32 can identify the source of WakeUp.
Problem description
Depending on the stm32 serie, configuration of the WakeUp pins differs:

the number of wakeup pin
GPIO pin  position, including a multiplexer 4 -->1
the trigger polarity : rising/falling edge
pull-up/down
the configuration (control) and status of each wakeUp pin is made through the Power (PWR) instance of each stm32.

Proposed change

Adding a new devicetree binding for each wakeUp pin (of each stm32 device)
Adding a new node for the PWR module of the stme32 device
Adding a new devicetree binding for using the WakeUp pins
Adding a software sequence to enable and configure each WakeUp pin in a new stm32_pwr module

Detailed RFC

The list of possible wakeup pins depends on the stm32 device from 1 to 8. Each WakeUp pin is known by its gpio reference example (PC13). The stm32 associates (hardware fixed) this GPIO pin to a wakeup pin number : example PC13 is wakeup pin 2.
Each possible WakeUp pin of the stm32 is a node of the stm32 soc devicetree.
Because some wakeup pin configurations depend on the stm32 serie, several bindings are defined (edge rising/falling detection, pull-up down (stm32h7), multiplexing (stm32u5).
When using a Wakeup pin, it is listed  in a node of the devicetree of the target platform.
During system Initialisation, the pwr module is parsing this list of wakeUp pins and configures the stm32 PWR registers.

Proposed change (Detailed)

dts/bindings/power/st,stm32-wkup-pinX.yaml  for the wakeup pin node of the stm32 soc devicetree
For example on the stm32g4 serie, the system wakeup pin 2 is the GPIO PC13:

wakeup2_pc13: wakeup2_pc13 {
		compatible = "st,stm32-wkup-pin1";
		#wkpin-cells = <1>;
		syswakeup-pin = <2>;
};

dts/bindings/power/st,stm32-pwr.yaml  for the pwr node of the stm32 soc devicetree
For example on the stm32g4 serie, the power node has 5 wakeup pins with configurable polarity

	pwr_wkup: pwr@40007000 {
		compatible = "st,stm32-pwr";
		reg = <0x40007000 0x400>; /* PWR register bank */
		clocks = <&rcc STM32_CLOCK_BUS_APB1 0x10000000>;
		pin-nb = <5>; /* 5 system WakeUp pins */
		pin-pol;  /* -->  the polarity of the pin for triggering is configurable */
		status = "disabled";
	};

dts/bindings/power/st,stm32-pwr.yaml  for the list (phandle-array) of  wakeup pins of the pwr_wkup node in stm32 board device tree
For example on the nucleo stm32g474re :

&pwr_wkup {
	status = "okay";
	wkpins = <&wakeup2_pc13 STM32_PWR_WAKEUP_RISING>;
};

A new module is introduced /soc/arm/st_stm32/common/soc_pwr_config.c
It configure and enable each wakeup pin listed in the wkpins property of the &pwr_wkup node.

Dependencies
power management : CONFIG_PM
--> Using the system wakeup pins is pertinent when the the stm32 supports the standby (PM_STATE_STANDBY)  and/or shutdown (PM_STATE_SOFT_OFF) low power modes.
Unresolved Questions
--> for each stm32 deice, the stm32XX.dtsi includes the complete list of wakeUp pins : that should be automatically generated in  the hal_stm32/dts/st/
--> Identify the wakeup source when exiting the low power (standby/shutdown) modes

One unresolved question is easy to answer: Identifying the wakeup source from shutdown mode.
It doesn't seem possible -- the WKUP flag registers are reset when exiting shutdown mode. So they're essentially useless...
https://community.st.com/t5/stm32-mcus-products/stm32l476-wakeup-flag-not-set-when-exit-from-shutdown-after/m-p/309420
Let me know if this isn't correct. This is the behavior I see on STM32L4P5 anyways.

One unresolved question is easy to answer: Identifying the wakeup source from shutdown mode.
It doesn't seem possible -- the WKUP flag registers are reset when exiting shutdown mode. So they're essentially useless...
https://community.st.com/t5/stm32-mcus-products/stm32l476-wakeup-flag-not-set-when-exit-from-shutdown-after/m-p/309420
Let me know if this isn't correct. This is the behavior I see on STM32L4P5 anyways.

Yes, does seem so. The best I was able to do was this:
https://github.com/zephyrproject-rtos/zephyr/blob/56720088cd57d18e828f98fb22a59f458a0375da/samples/boards/stm32/power_mgmt/wkup_pins_gpio/src/main.c#L39-L43


Rewrite to improve generalization

[x] Improve sync generalization: move common logic to the Api class to make adding new services easier
[x] Improve general generalization: to prevent having to repeat the service ID
[x] Isolate services: make sure that service implementations are isolated to their own folder, to prevent merge conflicts when merging PRs for different services
[x] Implement CLI script to create services: to make it much easier to add new services

Will rebace once #74 is merged.

Must be a lot of work to do these changes as the number of streaming services are increasing.

Hopefully it will save us work in the future.
That's actually another thing that I wanted to talk about. As the number of streaming services increases, so does the extension size, and considering that each service has its own content script file, this could lead to a rather large size later on. Do you think that we should just have one content script file for all services? This would increase the size of the file that has to be injected, but it would reduce the size overall.

That's actually another thing that I wanted to talk about. As the number of streaming services increases, so does the extension size, and considering that each service has its own content script file, this could lead to a rather large size later on. Do you think that we should just have one content script file for all services? This would increase the size of the file that has to be injected, but it would reduce the size overall.

Good question. I'm a bit worried about including a lot of unused code in each script. At least if they start interfering with each other. But I do see that the size seems alarmingly big already.  Not sure if it matters much that the size is large though. Might eat memory I suppose, and the initial download might be slower.
Good work with the script! Tested it now.

Moved the CLI to a separate repo (https://github.com/trakt-tools/cli) and added a command to update a streaming service as well.
So this PR is ready.


Move to Roslyn

Can you use any help on this?

You are welcome to make pull request for that :)

Nice.  A placeholder for moving to roslyn, just so I can add a +1  :)


Fix huber loss

TODO:

[x] merge and rebase #175

Hi @henry0312, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

rebased master.
However, there may be something wrong with ApproximateHessianWithGaussian.
I'll check.

@henry0312 This topic has a bunch of Gradient/Hessian functions for helping to converge for a L1 loss function when a hessian is mandatory for convergence.
They were all tested in xgboost and are all converging fast. Without a hessian, the convergence is slow (which seems to be the case in LightGBM also as @guolinke mentioned in the previous PR for L1 loss). It would be good to have an example to check the convergence speed of the L1 loss.
The short "explanation" is that when a hessian is mandatory for good convergence and we want to use a L1 loss function, we should avoid at any cost a constant hessian. To fix this issue, we smooth the linear function using a curve so the hessian is never zeroed out (better and faster convergence speed). For instance, the Fair objective (from a Microsoft research) is used to approximate such L1 function requiring a hessian.
The R versions with their gradient/hessian formulas are below:
ln_cosh_obj <- function(preds, dtrain) {
  labels <- getinfo(dtrain, "label")
  grad <- tanh(preds-labels)
  hess <- 1-grad*grad
  return(list(grad = grad, hess = hess))
}

ln_expexp_obj <- function(preds, dtrain) {
  labels <- getinfo(dtrain, "label")
  x <- preds-labels
  grad <-  (exp(2*x)-1) / (exp(2*x)+1)
  hess <-  (4*exp(2*x)) / (exp(2*x) + 1)^2
  return(list(grad = grad, hess = hess))
}

cauchyobj <- function(preds, dtrain) {
  labels <- getinfo(dtrain, "label")
  c <- 2  #the lower the "slower/smoother" the loss is
  x <-  preds-labels
  grad <- x / (x^2/c^2+1)
  hess <- -c^2*(x^2-c^2)/(x^2+c^2)^2
  return(list(grad = grad, hess = hess))
}

fairobj <- function(preds, dtrain) {
  labels <- getinfo(dtrain, "label")
  c <- 2  #the lower the "slower/smoother" the loss is
  x <-  preds-labels
  grad <- c*x / (abs(x)+c)
  hess <- c^2 / (abs(x)+c)^2
  return(list(grad = grad, hess = hess))
}

@Laurae2 the hessian of mae is delta function, therefore, if we can approximate it with something appropriate (e.g. gaussian function), converge and performance will be good, I guess.

I finished fixing ApproximateHessianWithGaussian (also Huber loss).
L1 loss is approximated with Gaussian function like http://www.wolframalpha.com/input/?i=Integrate[Erf[x]],+Integrate[Erf[x%2F10]].

Next, I will create a Fair loss in an another PR, which @Laurae2 tells me.


[prometheus] update recording and alerting

Enhancement

add introduction of Prometheus
highlight one-click setup
rename resources to be more practical

all done, close


Update NodePool.Status with node drift information.

Description
What problem are you trying to solve?
Currently, there is no clear way to identify how many nodes have drifted from the current hash (nodepool hash and ec2nodeclass hash). To determine the node rotation progress, we will have to look into individual node objects, nodeclaims, nodepool and nodeclass.
Since Karpenter controller identifies and rotates the drifted nodes, I am assuming the controller already maintains the list of drifted nodes (if not, identifies the drifted nodes in every reconciliation.)  It will be helpful to surface this information in the NodePool status.
for ex:
status:
  resources:
    cpu: "64"
    ephemeral-storage: 134205420Ki
    memory: 258565188Ki
    pods: "640"
    nodes:
      totalNodes: 10
      driftedNodes: 2

How important is this feature to you?
This feature will be very useful to identify the progress of node rotation whenever we change AMIs or trigger any other form of upgrades by updating the nodepool or ec2nodeclass.

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Does a metric work for you here? Or would you like a constant update of rolled-up information directly in the status? This is definitely something we are thinking about as we are thinking about how we can improve our observability of Karpenter for v1.

Does this request belong in the kubernetes-sigs/karpenter repo since it's about the netural concept of drift?

It would be advantageous to have this data accessible in both metrics and CR status. Adding it to the CR status would greatly benefit other watchers in the cluster,  We should also consider adding the number of Nodes that are restricted because of PDB violations.

It would be advantageous to have this data accessible in both metrics and CR status

We do have to be a little thoughtful about the number of updates that this would generate. I'm not saying that it's out of the question, but a metric is a bit easier to swallow only because they're pull-based and not push-based.

there is no clear way to identify how many nodes have drifted from the current hash

You could take a look at the NodeClaim status conditions to see if a NodeClaim has a "Drifted" condition. Counting these up across the cluster (or by label) should give you the info you want. Yeah, you have to construct it, but given that this doesn't currently exist in Karpenter, this is a possible workaround.


[BUG] distinct rule does not work

Prerequisites

[x] Checked if your Laravel Excel version is still supported: https://docs.laravel-excel.com/3.1/getting-started/support.html#supported-versions
[ ] Able to reproduce the behaviour outside of your code, the problem is isolated to Laravel Excel.
[x] Checked that your issue isn't already filed.
[x] Checked if no PR was submitted that fixes this problem.
[x] Filled in the entire issue template

Versions

PHP version: 7.2.19
Laravel version: 6.18.3
Package version: 3.1.19

Description
the distinct validation rule does not work as expected. It does not create an error on duplicate value.
Steps to Reproduce
Attempt to use distinct Laravel validation rule on a CSV column. My rules are generated dynamically, but this is the output of the rules() method of my MyImport class:
[
  "*.institution_code" => "alpha_num|size:5"
  "*.empl_id" => "numeric|digits:10|distinct"
]

I have also tried the rule without the leading *. and the result is the same
Expected behavior:
When there are duplicate values in the column (in this case empl_id), Laravel-Excel should create a failure error
Actual behavior:
No failure error is created/thrown to be handled in onFailure method.
Additional Information
Import Class
class GenericFileUploadDataImport implements
	ToModel,
	WithChunkReading,
        WithHeadingRow,
	WithValidation,
	SkipsOnError,
	SkipsOnFailure
{
	use Importable;

	public function chunkSize(): int
	{
		return 1000;
	}

	public function model(array $row)
	{
		$this->model_data = array();

		foreach ($this->template->fields as $field) {
			$this->model_data[strtolower($field->name)] = $row[strtolower($field->name)];
		}

		return new M\GenericFileUploadData($this->template->table_name, $this->model_data);
	}

	public function rules(): array
	{
		$this->model_validation = array();

                //SNIP
		//dynamically create model validation rules as above
                //END

		return $this->model_validation;
	}

	public function onFailure(Failure ...$failures)
	{
		foreach ($failures as $failure) {
			M\FileUploadError::create([
				'file_upload_id' => $this->file_upload_id,
				'row_number'	=> $failure->row(),
				'column_name'	=> $failure->attribute(),
				'errors'	=> json_encode($failure->toArray()),
				'row_values' => json_encode($failure->values()),
			])->save();
		}
	}

	public function onError(\Throwable $e)
	{
		throw $e;
	}
}

The validator in this case doesn't know about the contents of the entire file, only about the current chunk

@patrickbrouwers I have set my chunkSize to 1000 and my file only has 10 rows (last 2 are duplicated) but it still does not work.

@patrickbrouwers I switched my import from ToModel to ToCollection. The distinct rule is working when using ToCollection but does not work when using ToModel. Is this a bug in ToModel or expected behavior?

Expected behavior

I've updated the docs to reflect this caveat. Maatwebsite/laravel-excel-docs#97

@patrickbrouwers @juventus18 are you guys sure it doesn't work for ToModel concern?
In my case it works:
public function rules(): array
    {
        return [
            'number' => 'required|distinct:*.number',
        ];
    }

What am I doing wrong?

If it works you are not doing anything wrong ;)
I guess the difference is that you use *.number.
If you are sure it works, perhaps you can update that node in the docs with how you use it instead?

Apparently, it's because I'm using WithBatchInserts concern. Sent a PR.

Yes, that makes sense. Without batch inserts, the validator doesn't know about other rows, as it gets validated one by one

Hm... faced a very-very strange problem. distinct validation fails only on some strings. For example, these two number columns  are not distinct:
number,description,note,unit,interchangeable_with,parents,children,services
8E-1042,CLIP (SLOT),,piece,,"74,1007,1108,1311,5755,6166",,
8E-1043,CLIP (TAB),,piece,,"75,1008,1109,1312,5756,6167",,

    public function rules(): array
    {
        return [
            'number' => 'required|distinct:*.number',
            'unit' => 'required|in:piece,meter,liter',
        ];
    }

Validation fails when WithBatchInserts is switched on. I have absolutely no idea why this happens. Why does it happen only on these particular numbers and not others? And how does WithBatchInserts break it?

It validates in batches, so it will only be able to check distinct on the current batch; it doesn't know about the rest of the file as that's not read yet.  These 2 might be in 2 different batches.

But in this case, there should be false-negative results (everything is unique in one batch, that doesn't guarantee there are no duplicates between batches), not false-positive.
Moreover, validation of this particular file with only 2 lines fails with "not distinct" error.

I'm not sure then. You should probably try to reproduce this with the Laravel validator without and see if it gives the same error. We just pass it on to the validator, so not sure why it would fail.


backends do not load in proper order

This line has a comment # Iterate in reverse order of preference. However, it loads them with some random order, because the object to iterate is a set, which doesn't keep the order. In my Linux computer, when I run import arrayfire as af, the function will be called and it loads "" first, and due to err!=0, it directly returns and not loads anything.
By the way, I simply change {"cpu", "opencl", "cuda", ""} to ["cuda"] and I can import arrayfire with no error.

This is a mistake in the dev branch, the master branch should still have the correct code. Thank you for catching that! I'll make sure to address if we decide merge dev into master. There are a few incoming changes (related to library.py) that need to go into effect before we will be looking at the dev branch.


Feature: edit data files with a list/array at root level

- Do you want to request a feature or report a bug?
Feature
- What is the current behavior?
it seems to be impossible to create data files with a list at root level:
- nav_object_1
- nav_object_2

The current work-around is to use
navigation:
  - nav_object_1
  - nav_object_2

However, this may require a change of the template that may be part of libraries (jekyll theme gems).
- What is the expected behavior?
I can just create lists also on root level.

Yeah, everything has to have a field name currently. I suppose we could have a boolean "root" property on fields that simply drops the field's value directly into the parent (root for any field not in an object or list widget).

Would you be up for taking a swing at this?

Unfortunately I'm super busy with university. :(

No problem! I'm sure someone will take it on.

I would be happy to implement this - currently it is not possible to use gatsby-transformer-yaml with netlify since the transformer only supports lists.
Any pointers on where to look in the code to get started? I'm pretty new to netlify.

I have solved this on the gatsby side with a custom yaml transformer. I would still like to support it here for maximum flexibility :)

Forget the boolean thing I said before, we should apply the field/fields approach from the list widget to the collection config.
fields vs. field
Here's the list widget using fields:
# list field config using `fields`
- name: my-list
  label: My List
  widget: list
  fields: [{ name: title, label: title, widget: string }]

# output:
# list: [
#   { title: 'a title' },
#   { title: 'another title' }
# ]

Here's the same widget config using field:
# list field config using `field`
- name: my-list
  label: My List
  widget: list
  field: [{ name: title, label: title, widget: string }]

# output
# list: [
#   'a title',
#   'another title'
# ]

Are there any updates regarding the proposals mentioned above? This is a deal-breaker for Gatsby users, please see #1282.

@kripod I believe this is a feature the CMS should support, Until then, it is NOT a deal-breaker for Gatsby users when it is still able to map to a structure the CMS does support at the moment.

This is still happening.

I've just patched this locally in netlify-cms-core/src/backend.ts functions entryToRaw and entryWithFormat. Around the calls to either format.toFile or format.fromFile I'm just doing a wrap/unwrap with a special key rootArray.
  entryWithFormat(collectionOrEntity: unknown) {
    return (entry: EntryValue): EntryValue => {
      const format = resolveFormat(collectionOrEntity, entry);
      if (entry && entry.raw !== undefined) {
        let data = (format && attempt(format.fromFile.bind(format, entry.raw))) || {};
        let data = Array.isArray(data) ? { rootArray: data } : data; //Added wrapper for root arrays
        if (isError(data)) console.error(data);
        return Object.assign(entry, { data: isError(data) ? {} : data });
      }
      let data = format.fromFile(entry);
      return Array.isArray(data) ? { rootArray: data } : data; //Added wrapper for root arrays
    };
  }

  entryToRaw(collection: Collection, entry: EntryMap): string {
    const format = resolveFormat(collection, entry.toJS());
    const fieldsOrder = this.fieldsOrder(collection, entry);
    let data = entry.get('data').toJS();
    let data = data.rootArray || data; // Unwrap root array (this should probably check the fields.length == 1 too)
    return format && format.toFile(data, fieldsOrder);
  }

Not as clever as the approach mentioned above - but if you think of it as normalising files to always be a map instead of sometimes passing arrays around then it feels a bit less hacky.

No fix on this since 2017? )=

Hi @cyonder, we try to get to the most up voted features first. You can see the sorted list here https://github.com/netlify/netlify-cms/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc.
Regardless, we would love a contribution for this.

I just tried to see if I could get the ball rolling on this issue.

Parse the single entry file to get array of content
Based on that content, generate new  "real" entries
Populate the collection with previously generated entries

Doing edits / adding new content would be similar to above but in reversed order.
But the assumptions of content being in separate files made it very complicated straight away but maybe my idea was overly complicated?
Would love to get some help / discussion with main contributors to the project. <3

Hi @reimertz, thanks for pushing this forward.
What do you think about putting a draft PR for it? We can use it for the discussion.
Another option is our #contributing channel on Slack https://www.netlifycms.org/chat.

Hi everyone, any update on this ?
This seems blocking for Eleventy global data files too, if anyone has a workaround for this I'm interested :sweat_smile:

Hi everyone, any update on this ?
This seems blocking for Eleventy global data files too, if anyone has a workaround for this I'm interested :sweat_smile:

Hi @vnourdin this would make a great contribution to the CMS if anyone would like to pick it up

Hi @vnourdin this would make a great contribution to the CMS if anyone would like to pick it up

Just to said this is also an issue for @nuxt/content with array at the root level of a json. Like here.

+1 would love to see support for .json files with array at root level to mimic a MongoDB collection

would love to see support for this

Still needed :)

NEEEEED

Need it....

Is there any chance for this to be implemented anytime soon?

Here is a workaround:
            CMS.registerCustomFormat('flat-json', 'json', {
                fromFile: text => {
                    const body = JSON.parse(text) as Record<string, Record<string, string>>;
                    // Now we want to transform this object into a flat object with a 'key' field
                    const value = { items: Object.entries(body).reduce((acc, [key, value] : [
                        string,
                        Record<string, string>
                    ]) => {
                        acc.push({ key, ...value });
                        return acc;
                    }, [] as Record<string, string>[]) };
                    return value;
                },
                toFile: (value : { items: Record<string, string>[] }) => {
                    const body = value.items.reduce((acc, { key, ...rest }) => {
                        acc[key] = rest;
                        return acc;
                    }, {} as Record<string, Record<string, string>>);
                    return JSON.stringify(body, null, 2);
                }
            });

  - name: 'redirects-collection'
   label: 'Redirects'
   format: 'flat-json'
   files:
   - label: 'All Redirects'
     file: 'sigpwny.com/src/redirects.json'
     name: 'redirects'
     fields:
       - {label: 'Links', name: 'items', widget: 'list', fields: [
           # must be named 'key' for our custom formatter to work
           {label: 'From', name: 'key', widget: 'string', hint: 'Example: "/qrcode"'},
           {label: 'To', name: 'destination', widget: 'string', hint: 'Example: "myurl.com/foo"'},
           {label: 'Status Code', name: 'status', widget: 'hidden', default: 302}
         ]}


Added CI/CD for the Docusaurus Website

Motivation
https://issues.jboss.org/browse/AEROGEAR-10150
What
Added CI/CD for the Docusaurus Website

@ziccardi Does GH actions have a ci/cd status badge for the readme?

@secondsun yes!


403 Seektable Error

I'm encountering a "seektable" error specifically when downloading AAC quality songs on both macOS and Windows platforms. While using the --config-path option resolves the issue on Windows, it unfortunately doesn't work on macOS. This inconsistency between platforms highlights a potential bug or limitation in the current implementation.
bash-3.2$ votify https://open.spotify.com/playlist/0dzPh9MYzz7E7YJ6BZHkvM?si=j__xatQFRCC7OuPvEGk3eA
[INFO     23:53:43] Starting Votify
[INFO     23:53:44] (URL 1/1) Checking "https://open.spotify.com/playlist/0dzPh9MYzz7E7YJ6BZHkvM?si=j__xatQFRCC7OuPvEGk3eA"
[INFO     23:53:45] (Track 1/18 from URL 1/1) Downloading "if..."
[ERROR    23:53:47] (Track 1/18 from URL 1/1) Failed to download "if..."
Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 8, in check_response
    response.raise_for_status()
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://seektables.scdn.co/seektable/a70ffd875f433c855589c3f3103daa10bf2ac965.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/cli.py", line 597, in main
    downloader_song.download(
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 137, in download
    self._download(*args, **kwargs)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 165, in _download
    stream_info = self.get_stream_info(gid_metadata, "track")
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_audio.py", line 86, in get_stream_info
    seek_table = self.downloader.spotify_api.get_seek_table(file_id)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/spotify_api.py", line 278, in get_seek_table
    check_response(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 10, in check_response
    _raise_response_exception(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 14, in _raise_response_exception
    raise Exception(
Exception: Request failed with status code 403:
[INFO     23:53:52] (Track 2/18 from URL 1/1) Downloading "春よ、来い"
[INFO     23:54:02] (Track 3/18 from URL 1/1) Downloading "Pale Rain"
[INFO     23:54:11] (Track 4/18 from URL 1/1) Downloading "summertime"
[INFO     23:54:20] (Track 5/18 from URL 1/1) Downloading "ロビンソン"
[INFO     23:54:29] (Track 6/18 from URL 1/1) Downloading "美しい鰭"
[INFO     23:54:38] (Track 7/18 from URL 1/1) Downloading "strangers"
[INFO     23:54:47] (Track 8/18 from URL 1/1) Downloading "光るなら"
[INFO     23:54:55] (Track 9/18 from URL 1/1) Downloading "打上花火"
[INFO     23:55:04] (Track 10/18 from URL 1/1) Downloading "Prologue"
[INFO     23:55:13] (Track 11/18 from URL 1/1) Downloading "Skies Forever Blue"
[INFO     23:55:23] (Track 12/18 from URL 1/1) Downloading "アイデア"
[INFO     23:55:33] (Track 13/18 from URL 1/1) Downloading "アイネクライネ"
[INFO     23:55:44] (Track 14/18 from URL 1/1) Downloading "Mayonaka no Door / Stay With Me"
[INFO     23:55:53] (Track 15/18 from URL 1/1) Downloading "Plastic Love"
[INFO     23:56:03] (Track 16/18 from URL 1/1) Downloading "どんなときも。"
[ERROR    23:56:04] (Track 16/18 from URL 1/1) Failed to download "どんなときも。"
Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 8, in check_response
    response.raise_for_status()
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://seektables.scdn.co/seektable/720f41baa890a111f93a02f79ff36de0f0b13a8d.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/cli.py", line 597, in main
    downloader_song.download(
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 137, in download
    self._download(*args, **kwargs)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 165, in _download
    stream_info = self.get_stream_info(gid_metadata, "track")
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_audio.py", line 86, in get_stream_info
    seek_table = self.downloader.spotify_api.get_seek_table(file_id)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/spotify_api.py", line 278, in get_seek_table
    check_response(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 10, in check_response
    _raise_response_exception(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 14, in _raise_response_exception
    raise Exception(
Exception: Request failed with status code 403:
[INFO     23:56:09] (Track 17/18 from URL 1/1) Downloading "どんなときも。 - バラード・ヴァージョン"
[ERROR    23:56:11] (Track 17/18 from URL 1/1) Failed to download "どんなときも。 - バラード・ヴァージョン"
Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 8, in check_response
    response.raise_for_status()
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://seektables.scdn.co/seektable/18a8bb0d007246d2b123713bb39af7513cb485c3.json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/cli.py", line 597, in main
    downloader_song.download(
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 137, in download
    self._download(*args, **kwargs)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_song.py", line 165, in _download
    stream_info = self.get_stream_info(gid_metadata, "track")
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/downloader_audio.py", line 86, in get_stream_info
    seek_table = self.downloader.spotify_api.get_seek_table(file_id)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/spotify_api.py", line 278, in get_seek_table
    check_response(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 10, in check_response
    _raise_response_exception(response)
  File "/Users/akshat/Library/Python/3.9/lib/python/site-packages/votify/utils.py", line 14, in _raise_response_exception
    raise Exception(
Exception: Request failed with status code 403:
[INFO     23:56:16] (Track 18/18 from URL 1/1) Downloading "Young Girl A"
[INFO     23:56:21] Done (3 error(s)).6MiB ETA:1s]

Here's my config
{
    "wait_interval": 5,
    "enable_videos": false,
    "download_music_videos": false,
    "download_podcast_videos": false,
    "force_premium": false,
    "log_level": "INFO",
    "no_exceptions": false,
    "cookies_path": "/Users/akshat/.votify/cookies.txt",
    "output_path": "/Users/akshat/Music/Spotify",
    "temp_path": "temp",
    "wvd_path": "/Users/akshat/.votify/device.wvd",
    "aria2c_path": "aria2c",
    "ffmpeg_path": "ffmpeg",
    "mp4box_path": "mp4box",
    "mp4decrypt_path": "mp4decrypt",
    "packager_path": "packager",
    "template_folder_album": "{playlist_title}",
    "template_folder_compilation": "{playlist_title}",
    "template_file_single_disc": "{isrc}",
    "template_file_multi_disc": "{isrc}",
    "template_folder_episode": "Podcasts/{album}",
    "template_file_episode": "{track:02d} {title}",
    "template_folder_music_video": "{artist}/Unknown Album",
    "template_file_music_video": "{title}",
    "template_file_playlist": "Playlists/{playlist_title}",
    "date_tag_template": "%Y-%m-%dT%H:%M:%SZ",
    "save_cover": false,
    "save_playlist": false,
    "overwrite": false,
    "exclude_tags": null,
    "truncate": null,
    "audio_quality": "aac-medium",
    "download_mode": "aria2c",
    "remux_mode_audio": "mp4decrypt",
    "lrc_only": false,
    "no_lrc": true,
    "video_format": "mp4",
    "remux_mode_video": "ffmpeg"
}

macOS 15.1
Python 3.9.6
Votify 1.4.5

Hi,
I personally think that this is some kind of a blocking mechanism after two much requests and it temporarily disables access. For me the issue was always fixed after opening the link in my browser and then I could start downloading again. Maybe the project owner could implement a re-try mechnism and for example allow for an option that when checked would open up the seektable URL with the systems default browser, wait 5 seconds, close the browser and then try downloading it again.
But this is just a tip from my experience. Maybe I am totally wrong here xD

I am using windows 11 and getting the same error
how did you solve the win error what does --config-path mean?

someone mentioned for them it worked not for me.


dotnet publish

How can use the dotnet publish command?

@FalkWinkler This SDK does not support the dotnet command (other than for creating a new project using the template) as it is all NETFramework based. You can use MSBuild to build and publish your project.
msbuild /t:Publish /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingeFile=true /p:PackageLocation="MyPublishDir"

or
msbuild "path\to\your\solution.sln" /t:rebuild ^
				    /p:Configuration=Release ^
				    /p:DeployOnBuild=True ^
				    /p:DeployDefaultTarget=WebPublish ^
				    /p:WebPublishMethod=FileSystem ^
				    /p:DeleteExistingFiles=True ^
				    /p:publishUrl="path\to\your\output\folder"

These properties are documented for MSBuild and Publishing Profiles for ASP.NET 4 WebApplication projects.


Possible to Ack incomming client EmitWithAck messages?

If a client sends a EmitWithAck message, how can I send a response to ack the message from the go server?

If a client sends a EmitWithAck message, how can I send a response to ack the message from the go server?

that's through a callback, Wafje

socketClient.On("new-delivery-request", func(receivedData ...any) {
			fmt.Println("got new delivery request")
			receivedMessage := receivedData[0].(map[string]any)
			roomId := receivedMessage["socketRoomId"].(string)
			fmt.Println(receivedMessage)

			var callbackFn func([]interface{}, error)
			if fn, ok := receivedData[len(receivedData)-1].(func([]interface{}, error)); ok {
				fmt.Println("Callback is a function!")
				callbackFn = fn
				mapData := map[string]interface{}{
					"hasInterest": true,
					"who":         "wafje",
				}
				jsonData := []interface{}{mapData}
				callbackFn(jsonData, nil)

			} else {
				fmt.Println("Callback is not a function")
			}
}

the last payload from any event that has an acknowledgement is always a function with the signature " func([]interface{}, error)", invoke it like a callback function same way i did in the code above...and the emitter will get the response.


support go 1.17

Description
Homebrew had to revert to go1.16 in https://github.com/Homebrew/homebrew-core/pull/84487 due to an issue with x/sys.
#715 attempted to build with go1.17, but it seems tests failed.
Expected Behavior
To successfully build with go1.17
Additional context
https://github.com/Homebrew/homebrew-core/pull/84487

TODOs

[ ] Undo canaries updates after testing

Waiting for this to resolve - https://github.com/josephspurrier/goversioninfo/issues/57

Waiting for this to resolve - josephspurrier/goversioninfo#57

I added a comment here - let me know if that resolves it: https://github.com/josephspurrier/goversioninfo/issues/57#issuecomment-1105319291


changed the navbar

Closes #40
Closes #29

LGTM! Smooth scrolling will look better for the home button click action.

LGTM!


Data import: maintenance mode during data update

There's now a mechanism in place to automatically refresh data by downloading from GBIF.
During this process, the database spends a few minutes in a messed up state (occurrence duplicates because new data is added before the previous one is deleted, ...).
We need to show the website as temporarily unavailable during this time to avoid presenting incorrect data to users. I consider using django-maintenance-mode for that.
Additional considerations:

[x] the data imports will probably be run at night, to limit interference with the users
[ ] infrastructure: we'll need an alert (mail? others) for urgent intervention if the website stays for too long in maintenance mode (error during the data import process: GBIF outage, bug, ...) (not needed thanks to transactions)
[x] we need to take precautions so user alerts (mail for new observations, ...) are not sent when the website is in maintenance mode

I definitely agree with you.
If we run the data import at night (I would say midnight) as you said AND the early warning system alerts are sent at 2pm for example, then we have time to check what is going wrong.

Yeah. But I'll also implement also a check in the "sending alert" code to make sure they just it just waits if maintenance mode is on (so we don't break everything because we decided to change hours or because the server time is incorrect)

Yes, for sure! 👍

The occurrence import is now atomic (#62). If all goes well, we should be able to tackle this simply and elegantly using the following logic:

new data is downloaded from GBIF
maintenance mode is set to ON (website inaccessible + no emails sent)
the occurrence import is done (atomically: all changes or nothing in the database)
if the transaction was aborted: send an email to administrators to check what happened
in all cases, the maintenance is set to OFF (in worse case - transaction aborted - no new data will be visible in the website but at least the data is still perfectly consistent and the website isn't stuck in maintenance mode)

I find it a good workflow, it's defensive enough, I think. One small question, maybe trascurable problem: what to do with downloaded data in 1 if 4 happens for many consecutive days (e.g. you and your hpothetic back-up person are both on holidays)? Probably memory it's not a big issue, right?

Indeed, I don't expect any disk space problem:

The downloads are quite small (with the current species list, we're at 7 Mb / day)
In addition to that, the download happens in a temporary directory: as soon as the processes finish, the system knows this disk space can be freed when necessary.

Update: I'm considering using https://github.com/fabiocaccamo/django-maintenance-mode for the maintenance mode itself.

This is working fine for years, IU think we can close this!


Do context.refresh only works with FutureProvider ?

My StateNotifierProvider is:
final homepageStateNotifierProvider =
    StateNotifierProvider<HomepageStateNotifier>((ref) {
  final homepageRepo = ref.watch(homepageRepoRrovider);
  return HomepageStateNotifier(homepageRepo);
});

and I want to refresh this provider using pull to refresh so my widget is:
 RefreshIndicator(
        onRefresh: () => context.refresh(homepageStateNotifierProvider),
        child: MyChildWidget()
}

but I am not able to do this and gives error:
The argument type 'StateNotifierProvider<HomepageStateNotifier>' can't be assigned to the parameter type 'RootProvider<Future<void>, Object>'.

Is context.refresh only available for FutureProviders or there is another way to do this?

context.refresh works with everything. The problem isn't  context.refresh, but RefreshIndicator, which wants a Future to keep the indicator visible while the data refreshes.
Assuming your StateNotifier uses AsyncValue like so:
class MyNotifier extends StateNotifier<AsyncValue<Model>> {
  MyNotifier(): super(const AsyncValue.loading());
}

you can do:
 RefreshIndicator(
  onRefresh: () => context.refresh(stateNotifierProvider).stream.firstWhere((value) => value is! AsyncLoading),
  child: MyChildWidget()
}

You assumed exact what I was doing. Thank you very much.

@rrousselGit

context.refresh works with everything.

Does it work with Provider.autoDispose.family? I am getting the error:
The argument type 'AutoDisposeProvider<String?>' can't be assigned to the parameter type 'RootProvider<dynamic, Object>'.```

@derolf, Yes it works, would you provide some code snippet what you are trying to do and how?


feat: clip memory for Lambda if it exceeds context

AWS Accounts with AWS Lambda memory size limits below 7076Mb will fail deployments without adjusting.
Override the default configuration if necessary through the CDK context, https://docs.aws.amazon.com/cdk/v2/guide/context.html:
cdk deploy --context maximumLambdaMemorySize=3008

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

looks good, just some small queries.

I replied and resolved with no changes.


Allow disabling chrome smooth scrolling

I confirm (by marking "x" in the [ ] below):

[x] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[x] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[x] I have read contributing guidelines.

Summary
The desktop client forces smooth scrolling and does not provide a way to disable it.
For some reason I become very dizzy when an application uses smooth scrolling and have to quit using it, so this is indeed an issue for me and not a feature request.
Steps to reproduce
My setup consists of:

Archlinux
Mattermost desktop client version 3.4.1
Mattermost server version 3.5.1-1

Scroll in the desktop client.
Expected behavior
A possibility to disable smooth scrolling.
Observed behavior
No obvious possibility to disable smooth scrolling.
Possible fixes
Offer a way to visit chrome://flags inside the desktop app or provide an option in settings to disable smooth scrolling.
I'd also be very happy if I could disable it through the developer tools, config file, eg. something that is not patching and recompiling.

Possibly Chromium's command line switches are applicable. http://peter.sh/experiments/chromium-command-line-switches/
Does --disable-smooth-scrolling have any effects?

Yes indeed, thank you very much.
This issue can be closed if you are okay with this solution (I am).

Thanks @BotoX :)
@yuya-oc The command line switches seem pretty comprehensive, wondering if we should add a reference somewhere in our docs?

@jasonblais Yeah, it would make sense.

I was reading through the list and I ended up being a bit worried about referencing it in our docs before we've had a chance to test if they work fine with the app. I worry there might be a command that causes issues that we cannot troubleshoot.
So I think for now it's okay to reference them but I won't add them to our docs yet,

Closing the issue as resolved


build/mobile: RBE fails with arm64 versions of Bazel on macOS

Title: build/mobile: RBE fails with arm64 versions of Bazel on macOS
Description:
When building something in the mobile/ workspace with an arm64 bazel binary on an M1 Mac with RBE enabled, the protoc with the wrong architecture is attempted to be run, leading to this error:

sandbox-exec: execvp() of 'bazel-out/.../com_google_protobuf/protoc' failed: Bad CPU type in executable

Repro steps:
On an M1 Mac with EngFlow certs installed at /path/to/certs/engflow.crt and /path/to/certs/engflow.key:
git clone https://github.com/envoyproxy/envoy.git
cd envoy/mobile
./bazelw build \
  --tls_client_certificate=/path/to/certs/engflow.crt \
  --tls_client_key=/path/to/certs/engflow.key \
  --config remote-ci-macos \
  //library/common:envoy_main_interface_lib

The build succeeds if this line gets hardcoded to amd64: https://github.com/envoyproxy/envoy/blob/367763e5c63c003e495981e98fd9967bbcb86d20/mobile/bazelw#L16
Admin and Stats Output:
N/A
Config:
N/A
Logs:
N/A
Call Stack:
N/A

protoc (toolchains) was updated here #24546 - not sure if i made some mistake there


New Apple Wireless Keyboard FN_L to CONTROL_L not working

is anyone able to map the left function key to control? it seems like none of the apps can do it... doublecommand totally hit the wall a while ago and karabiner seems to also not being able to do it.
the only work around I have atm is reinstalling doublecommand after ever logout/restart.
Karabiner seems to not work with the new keyboard that apple recently released. it works with my old, not with the new one.

#529


Deposit state transition front-running

An entity that can provide proof for fraudulent ECDSA signatures or SPV proofs in the liquidation flow is rewarded with part of the deposit contract ETH value.
However, the methods under which proof is provided are not protected from front-running allowing anyone to observe transactions to provideECDSAFraudProof/ provideSPVFraudProof and submit the same proofs with providing a higher gas value.

[ ] implement a commit/reveal scheme to lock in proof in one block, and reveal the details in another.

Gut reaction here was that we're comfortable with this tradeoff, at least for v1, but need to mull it further and haven't gotten to it yet.

we're comfortable with this tradeoff

Agree here

The note I sent to the auditors for the audit on this one:

In general, we are comfortable with front-runnable interactions that ensure system integrity, as long as such front-running does not remove the original incentive of the submitter. We believe remaining front-runnable interactions have clear benefits to system actors, such that even if they are front-run, they have reason to submit the transaction.

Closing!


Need to pull upstream project to support newer language features. E.g. Text Blocks

Newer versions of the google/google-java-format project support Text Blocks and other newer Java language features. Right now when I try to use them, this plugin is outputting: error: unclosed string literal

I think related to this is also missing support for new style switch (namely comma-separated cases) https://dev.to/abh1navv/java-switch-case-the-improved-version-1012


repurpose this blog post content as a 'quickstart' section in the docs?

https://blog.fabric8.io/a-busy-java-developers-guide-to-developing-microservices-on-kubernetes-and-docker-98b7b9816fdf#.aerjlhhbd
as it kinda shows how to use the plugin from a java developers perspective who may know an app server or something etc.

@rhuss : We do need to update our README with a new blog, as per left Mojos or maybe use https://asciinema.org/ for showcasing a standard Java developer workflow using fmp.

@rohanKanojia agreed, I opened #1409 to track the README update, so lets close this one for now.


[GOAL] Subcoordinates stacked along Y-axis with 'group' support

Summary and Links

channel-type-grouping Support channel-type grouping with different sampling and amplitude range (normalization).

grouped-y-scaling: Manual scaling/zooming of Y axis per channel group

includes hover-based scroll-zooming per group

channel-group-yticks Switch y-ticks to group values when zoomed out enough that channel-based y-ticks are cluttered.. So a zoomed out view might be something like the following y-ticks (instead of have a tick per row):

https://github.com/holoviz/holoviews/issues/5901

groupwise normalization and zooming compelted


Disable day click

Feature request:
I'd like disable the click on the day number (in the month view) but change in day view when clikc on in everywhere in the box of the day (in month view). I tried setting in timespanClicked calendarView to 'day', the view change but the day is always today and not the clicked day. I need it also to have in the day view the events of the day (if i switch to the day view clicking on the day number the calendarCell is undefined).
Thanks

I've put together an example for you that shows how to implement this with the existing API, hope that helps :) http://plnkr.co/edit/ZNnLvvE60rUTkXGRVHll?p=preview

How I can fully disable day click? So events wont show off?

@qbunia https://mattlewis92.github.io/angular-bootstrap-calendar/#?example=slide-box-disabled


METAMODEL-1154: Added model for doing deletes and updates on table data

Fixes METAMODEL-1154

Merging based on lazy concensus


Hook after response

Is there a way to add a hook on a route or controller and do something with the response object?

Check this hook that I made to add a new method to the request object. Maybe it can help you: https://github.com/Josebaseba/sails-hook-validator

I think what your code does is review the request object, what I want to do is do something with the response object. So I don't want to do anything before the response, it's like adding a lifecycle afterUpdate or whatnot. But thanks anyway. I'll check this out for future reference.

  In the future for, please post support questions to our Google Group or Stackoverflow, or try our live Gitter chat room.
@Josebaseba your solution is good, but if all that is required here is a way to modify the response before it's sent out, we suggest customizing the "api/responses/ok.js" response and using that in your controllers (i.e. do res.ok() instead of res.send()).


$SET Compiler directives in cobol-sources should be ignored/processed

Some programs tend to have compiler directives in the very first line of a COBOL-source,
thus starting with $SET, the $ being positioned in the 7th column (like a comment).
The parser should either ignore such statements - treat them like a comment (first and easys tep) or consider the semantics of the various options (some options are very "doubtful" - like NOCHECKNUM, SPZERO etc etc and even influence the runtime behaviour)
2017-08-03 12:33:22.010 [main] ERROR com.csi.parser.CobolParseXXX.parseFile (79) - Cobolfile: XXXXASX.CBL threw exception: {}
java.lang.RuntimeException: Is FIXED the correct line format? Could not parse line 1:       $SET DIALECT(MF) NOTRUNC NOBOUND
	at io.proleap.cobol.preprocessor.sub.line.reader.impl.CobolLineReaderImpl.parseLine(CobolLineReaderImpl.java:64)
	at io.proleap.cobol.preprocessor.sub.line.reader.impl.CobolLineReaderImpl.processLines(CobolLineReaderImpl.java:100)
	at io.proleap.cobol.preprocessor.impl.CobolPreprocessorImpl.readLines(CobolPreprocessorImpl.java:110)
	at io.proleap.cobol.preprocessor.impl.CobolPreprocessorImpl.process(CobolPreprocessorImpl.java:99)
	at io.proleap.cobol.preprocessor.impl.CobolPreprocessorImpl.process(CobolPreprocessorImpl.java:86)
	at io.proleap.cobol.preprocessor.impl.CobolPreprocessorImpl.process(CobolPreprocessorImpl.java:65)

Thanks, fixed in fb735cf214d2a0424996b2f7f0c72716d989ac94. Added a unit test. With this fix, lines with a $ line indicator are blanked by the preprocessor, i. e. ignored by the parser.
Some refactoring in the preprocessor would be necessary to provide the original lines/compiler directives to the ASG. The implementation has to be done together with fixing issue #41.


dlib with qualcomm snapdragon math libraries

Hi Davis,
I'm trying to replace dlibs interface to blas and lapack with the qualcomm optimized QSML library. It shouldn't be too difficult since the library does provide all standard blas/cblas and  lapack interfaces. I've edited the CMakeLists.txt in dlib folder for it to use the include path and library path from the qsml library but where all should I change the code to include the new headers instead of the old ones. Once done, I can commit the changes upstream for you to review them.
Regards
Sahil

You only need to modify this one cmake file https://github.com/davisking/dlib/blob/master/dlib/cmake_utils/cmake_find_blas.txt.  It's just a matter of making it output what library you want to link with.  Since the BLAS API is essentially a standard there are no headers or anything else to mess with.

Thanks for your quick response. I pretty much did the same thing with using 2 new variables DLIB_USE_QSML_BLAS and DLIB_USE_QSML_LAPACK and modified the code accordingly. The thing is that in my include folder the headers have names like netlib_lapack.h, qblas1.h, qblas2.h.... I was wondering if I needed to update the code in dlib somewhere to make it use these headers or just specifying the folder name in a new file called cmake_find_qsml_blas.txt should be enough.

You don't need any new cmake options nor do you need to worry about those
headers.
I'm not familiar with QSML, but if it's really a BLAS library then all that
is needed is to modify that cmake file I mentioned.  If it's a BLAS library
and you are modifying other dlib files then you are doing something wrong.

QSML provides all the standard blas 1 2 and 3 interfaces. I'll try to just update the cmake_find_blas.txt file instead of messing with CMake options. If it doesn't work, I'll hit you up for advice again. Thanks again for your wonderful work on dlib and your awesome support for the community.

No problem :)
All that's needed to work with a BLAS library is just to add the linker
statement.  It should look just like all the other instances in
cmake_find_blas.txt.  So you should only need to add a few lines of text to
that file.

@davisking One problem with libraries such as this is that they may not be supported at run-time (due to lack of Snapdragon chip in this instance). Is there provision to allow fallbacks?
The headers in this library provide a function to check if the library is supported at run-time.

No.  This is all fixed at compile time.  If you need both then you need to compile separate binaries and invoke or link to the right thing yourself.


PySpark integration

Need a new context (:py?)
Need to see how to integrate and use PySpark from scala.

That would be an awesome feature to have!


Issue target="_blank" , PyQt

Issue target="_blank" and  website in web-view is small ( zoomed out )
ISSUE 1
It does not open external links: with target="_blank"
ISSUE 2
Page zoomed out too much , not give same view as other browsers.
Hope to see update

@CodeX-All Would you like to work on this?

@CodeX-All Would you like to work on this?

Well i can help
For Issue 2 : it can be solved using this :
`# Set the QT_AUTO_SCREEN_SCALE_FACTOR environment variable
import os
os.environ['QT_AUTO_SCREEN_SCALE_FACTOR'] = '1'`
should be present before this, if  i'm correct
if __name__ == "__main__":
    main()

That should solve display for all screens in correct size of current device resolution
for Issue #1 , believe or not , this my first 3 days in python


Directory Structure Changes

FYI, this repository will be restructured to the same structure as https://github.com/kubesphere-extensions/ks-extensions.
Since some of the open source extensions still need a place to be stored, I suggest that we store them under the src folder
The rest of the CI and ks-extensions repositories will be the same, so that we can reuse as much code as possible.

/cc @Qi-ming-Zhang @webup


Sequence Support

A handy feature: https://www.postgresql.org/docs/8.1/static/sql-createsequence.html
Noticed it's not under value generation: https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/blob/3afbfcac5ace71f7a54a3e4ab37ad434351d5d11/doc/value-generation.md
And didn't notice anything that suggests this is hidden away. Sequences are handy in instances where I'd like to basically use a serial but start it at a predefined number. Serials actually use sequences but again -- I don't really have a way to tweak it easily with FluentAPI.
I'll probably be falling back on Sequences for now (not really mandatory right now to start above 1), but figured I'd open a ticket in case any other interest starts up on this.

It exists, just isn't documented:
https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/blob/e9e89f8e18345a4dd5e5e8d2e6dec8c0c61f8c58/test/Npgsql.EntityFrameworkCore.PostgreSQL.Design.FunctionalTests/ReverseEngineering/Expected/SequenceContext.expected
Maybe I'll make a PR to document it once I get it working! 😄

@StrangeWill sequence are documented in the general EF Core docs. The Npgsql docs generally only discuss things that are specific to Npgsql/PostgreSQL.
I'll add a sentence point to the EF Core docs, though.

Have added some documentation in https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/blob/master/doc/value-generation.md, this will go live soon.

Yeah I'm dumb -- should have paid closer attention and noticed it was an EF Core feature... shucks.


try to download artifact.

Hello,
I upload a foo-bar-1.0.jar file to my local machine repo, and write these script:
import repositorytools
from repositorytools.lib.artifact import Artifact, RemoteArtifact
url = 'http://localhost:8081/nexus/content/repositories/releases'
client = repositorytools.NexusRepositoryClient(repository_url=url, user='admin',password='admin123')
artifact = RemoteArtifact.from_repo_id_and_coordinates('Release', 'foo:bar:1.0:demo:jar')
print(artifact)
client.resolve_artifact(artifact)
print(client)
got the result of this statement:

print(artifact)

foo:bar:1.0:demo:jar
it seems not right, and the later statement

client.resolve_artifact(artifact)

report _send_json function failed.
it seems that the file in lib/artifact.py RemoteArtifact.from_repo_id_and_coordinates() can not resolve a json format . I change my code:
artifact = RemoteArtifact.from_repo_id_and_coordinates('Release', 'foo:bar:1.0')
but get the result:
foo:bar:1.0::
and still failed,
I'm not sure the root cause, i'm not quite familiar with requests module, Could you post some demo code about upload,  download and delete?

oh.. what a markdown format..

report _send_json function failed.
Please send a traceback if there is any

foo:bar:1.0:demo:jar
Traceback (most recent call last):
File "nexus.py", line 44, in
client.resolve_artifact(artifact)
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\li
\repository.py", line 87, in resolve_artifact
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\li
\repository.py", line 186, in _send_json
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\li
\repository.py", line 176, in _send
File "C:\Anaconda3\lib\site-packages\requests\models.py", line 808, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found

Instead of
url = 'http://localhost:8081/nexus/content/repositories/releases'

try
url =  'http://localhost:8081/nexus/'

and instead of
artifact = RemoteArtifact.from_repo_id_and_coordinates('Release', 'foo:bar:1.0:demo:jar')

try
artifact = RemoteArtifact.from_repo_id_and_coordinates('releases', 'foo:bar:1.0:demo:jar')

Can I close it?

oh.. Please hold on, I'm get home now, I will updated it tomorrow..
Great appreciate for your help!

Ok, no problem. Feel free to support my effort via donations on Gratipay https://gratipay.com/~stardust85

Hello:
I try to update the code, current code is:
import repositorytools
from repositorytools.lib.artifact import Artifact, RemoteArtifact
url = 'http://localhost:8081/nexus/'
client = repositorytools.NexusRepositoryClient(repository_url=url, user='admin',password='admin123')
artifact = RemoteArtifact.from_repo_id_and_coordinates('releases', 'foo:bar:1.0:demo:jar')
print(artifact)
client = repositorytools.repository_client_factory()
client.resolve_artifact(artifact)
print(client)
still get the eror message:
foo:bar:1.0:demo:jar
Traceback (most recent call last):
File "nexus.py", line 92, in
client.resolve_artifact(artifact)
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 87, in resolve_artifact
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 186, in _send_json
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 176, in _send
File "C:\Anaconda3\lib\site-packages\requests\models.py", line 808, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found
I notice the delete function is almost the same as download, so I try the upload,  in these code:
import repositorytools
from repositorytools.lib.artifact import Artifact, RemoteArtifact
url = 'http://localhost:8081/nexus/'
artifact = repositorytools.LocalArtifact(local_path='c:\work\oliver-1.2.3.jar', group='foo')
print(artifact)
client = repositorytools.repository_client_factory(repository_url=url,user='admin',password='admin123')
remote_artifacts = client.upload_artifacts(local_artifacts=[artifact], repo_id='releases')
print(remote_artifacts)
but get the error message like this:
foo:oliver:1.2.3::
Traceback (most recent call last):
File "nexus.py", line 53, in
remote_artifacts = client.upload_artifacts(local_artifacts=[artifact], repo_id='releas
es')
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 108, in upload_artifacts
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 131, in _upload_artifact
File "C:\Anaconda3\lib\site-packages\repositorytools-4.0.2-py3.4.egg\repositorytools\lib
\repository.py", line 176, in _send
File "C:\Anaconda3\lib\site-packages\requests\models.py", line 808, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed
it seems that the info which passed to is correct:
artifact = RemoteArtifact.from_repo_id_and_coordinates('releases', 'foo:bar:1.0:demo:jar')
print(artifact.repo_id)
I can get output: release and the other is correct,
it seems that is the request(i'm not sure), Could you please help me on this?
sorry for bringing inconvience.
Best regards
Oliver

In repository.py, resolve_artifact method, the string 'service/local/artifact/maven/resolve' is treated as path, i'm not sure it is valid..??

Probably your artifact is missing:
https://oss.sonatype.org/service/local/artifact/maven/resolve?g=org.codehaus.jtstand&a=jtstand-core&v=0.98&r=codehaus-releases (should show some xml or if you path accept:application/json than json)
https://oss.sonatype.org/service/local/artifact/maven/resolve?g=org.codehaus.jtstand&a=idontexist&v=0.98&r=codehaus-releases (should show 404)
Try debug mode (artifact -D resolve ...)

You can also try running system tests or compare the code in system tests with your code. Unfortunately it is not running on travis CI yet...

oh..Ok, I don't know hot to run it on command line, never mind, I will try it on..

You can run tests conveniently in vagrant as documented in https://github.com/stardust85/repositorytools/blob/master/docs/development.rst

oh..Ok, I get it,
I'm trying it on my private Nexus repo, the function which I wanted is quite simple, download , delete and upload via python code, I try to upload a jar file manually, now, I can notice it on browser and download it via this link:
http://localhost:8081/nexus/content/repositories/releases/foo/bar/1.0/bar-1.0-demo.jar,
I can download it via browser directly in python, but I have no idea how to upload and delete, so I choose to search related module of python on internet.

You can use the http methods PUT and DELETE. Don't forget authentication.

wow... thanks!!
how dizzy I am..

It's done, Please close this issue, Great appreciate


Newly generated profiles are broken

I am not sure whether it's backend or front end, but newly created profiles have a broken layout and people can't login with these profiles.

@bacharakis @alexdor I attempted to debug it locally but I faced some issues. I logged in with  new account and though the login page redirected me to localhost, the user was not logged in and not added in the backend, so I cannot reproduce the broken layout issue.

Did you delete your user from the admin app? if so you should be getting an error that you need to contact an administrator (check the url, the frontend doesn't display or check any error)

@alexdor I did delete the user from the admin app and I am getting the error.

Yeah, that’s because if the user is deleted (which means banned) we have to manually enable him again. Could you try logging in with a different user?
On 6 Mar 2019, 17.29 +0100, Iro Palampougiouki<EMAIL_ADDRESS>wrote:

@alexdor I did delete the user from the admin app and I am getting the error.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I did log in with two new users and cannot reproduce the issue. We had a regression yesterday which had this result on the user page and was fixed with pr #265 and #267. @bacharakis are those print screens from today?

Even if they are from today, it might be cached version (the service worker
helps a lot with the speed of the website but it creates a n+1 cache issue,
there are a few good points for the service workers here
https://gist.github.com/Rich-Harris/fd6c3c73e6e707e312d7c5d7d0f3b2f9 )
On Wed, 6 Mar 2019 at 17.54, Iro Palampougiouki<EMAIL_ADDRESS>wrote:

I did log in with two new users and cannot reproduce the issue. We had a
regression yesterday which had this result on the user page and was fixed
with pr #265 https://github.com/ossn/ossn-frontend/pull/265 and #267
https://github.com/ossn/ossn-frontend/pull/267. @bacharakis
https://github.com/bacharakis are those print screens from today?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ossn/ossn-frontend/issues/276#issuecomment-470186429,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AICthwEJ8m0D_NrYOXXbQ7MWGkTgbWLcks5vT_LXgaJpZM4bhDOl
.

I just checked with a clear cache and got this:


Add GROMACS support

#47
I uploaded gromacs data to test_data repository.
I uploaded here the notebook test_gromacs_support.txt where I tested function from_gromacs. It seems working fine, but I have some concerns:

I didn't add cache support to the function
It's a bit weird when extracting structure. Structure information starts from some line not lattice (see the notebook). But lattice works.
I'm using site_properties attribute of Trajectory to introduce residues
It could be cool to have opportunity to filter trajectory by residue (see the notebook)
I got out_of_memory problem when plotting msd from large simulation

msd.csv
test_gromacs_support.txt

Hi @AstyLavrinenko thanks for this. I fixed a few small issues with the code. The plot labels were indeed a bug that incorrectly assumed that trajectory stores the time_step in seconds instead of fs.
The out of memory issue is somewhat to be expected with such large systems, especially with lots of time steps.
I will add some tests and then this can be merged.


feat: Use relative movement packets

Use the correct relative movement packets and only send position updated when needed. Substantially decreases bandwidth and packet count.

Use the correct relative movement packets and only send position updated when needed. Substantially decreases bandwidth and packet count.

I don't see needs_resync being set to true anywhere. Is this planned to impl in the future?

Its not really needed rn. Just put it there as an api in case its needed in future.


Add some plugins for support php

Added Phpactor is a better tool implementation php for vim
Added php-cs-fixer a tool to automatically fix PHP Coding Standards

Thanks @afranioce !


Fix for Bug 510108: [Regression]Reality data picker shows all Reality…

… data irrespective of project footprint or Extends.
The 8 corners are calculated in the ECEF coordinate system but I think they should be calculated in the spatial coordinate system and then converted to ECEF.
This will create a box oriented in the spatial coordinate system (as oppose to actual code that oriented it in ECEF world, which give a larger range in this particular case where the elevation range is high)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@Mergifyio backport release/2.9.0 release/2.10.0


32-bit Windows installer version available?

Not at this time, you can use the -service flag to help orchestrate some of the pain. We will likely address these issues in a near term future release


[rootless] unable to delete files in another namespace (with --rm)

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
When creating files in a rootless container and chowning these files to another user, it's impossible to remove them from the host without help of the root user.
Steps to reproduce the issue:

podman run -ti --rm -v /home/podman/we:/we alpine
adduser tobwen
cd /we && touch 1 && chown tobwen 1
[leave container]
rm /home/podman/we/1

Describe the results you received:
File cannot be removed, since it belongs to a non-existing user:
$ ls -al
total 8
drwxr-xr-x  2 podman podman 4096 Mar 21 21:41 .
drwxr-xr-x 12 podman podman 4096 Mar 21 21:45 ..
-rw-r--r--  1<PHONE_NUMBER>35    0 Mar 21 21:41 1

Sure, this is connected to https://github.com/containers/libpod/issues/1386 but since the original container has been removed, there's no way to figure out the namespace it was created in.

The UID mappings should always be the same (assuming you did not modify /etc/subuid or /etc/subgid) so we should still be good to delete these files

(The modified subuid/subgid case is sort of concerning, but you need root to do that, so I'm somewhat OK with saying you need root to fix it)

There is also
https://podman.io/blogs/2018/10/03/podman-remove-content-homedir.html
The buildah method works.

@mheon They look like this:
$ cat /etc/subuid
tobwen:100000:65536
podman:165536:65536

$ cat  /etc/subgid
tobwen:100000:65536
podman:165536:65536

@neVERberleRfellerER Thanks let me try this solution (as mentioned in #1386, but would be great to have this working in podman the native way).

@tobwen The buildah unshare method ought to still work, since we join namespaces with the same UID/GID maps always, then.
Though it's not good that we can't do this natively with Podman. We shouldn't need an external tool that might not be available in order to clean up after ourselves.

@mheon Can't I do this with normale unshare from util-linux, too? Or is this a complete different approach?

Not 100% sure here - they are conceptually very similar, but the semantics might be different enough to make it impossible (I'd mostly be concerned with how unshare gets its user mappings configured).

Hmm... building buildah fails here right now. I'll report & close as soon as I got this working.

A small work-around (where 165635:165617):
addgroup --gid 165617 conuser
adduser --gecos "" --disabled-login --uid 165635 --gid 165617 conuser

@tobwen There is a hope that someone will implement podman unshare, but no one has had it prop to the top of the priority list yet.

@rhatdan But it helps to delete the files. But is it normal that buldah greps existing users? 165635 gets systemd-timesync here.

@tobwen buildah/podman are using newuidmap and newgidmap commands provided by the shadow-utils package.  These programs search for the username in the /etc/subuild and /etc/subgid files respectively.  When the find an entry for the user they set the usernamespace in the kernel with the range of UID's specified.

@rh-atomic-bot But my /etc/subuild only has 2 entries. Seems like there's an overlap? I've never modified these files.

No overlay.
tobwen gets UIDs<PHONE_NUMBER>35
powman gets the next 65536 starting at 165536

Yes, but the first user created inside the container gets 165635 and when accessing the namespace with buildah, this user gets systemd-timesync, which is 100 on the host.

Please explain exactly what you mean by the first user inside of the container
cat /proc/self/uid_map while in the container,  you should see the mapping.

@rhatdan Please read the initial post, I've explained exactly, what I've done (I've created a new user inside the container, the first new one after root).
Since the container doesn't exist anymore, I cannot rejoin it. So I cannot run cat /proc/self/uid_map in there, but the mapping is right in there, of course.
My initial problem was that I cannot remove files, but this works using unshare or buildah. I'm just curious, why buildah presets me systemd-timesync as a user, while unshare maps it to nobody

Any UID not mapped into the User Namespace will be referred to as nobody.

@rhatdan So Buildah unluckely maps the namespaces UID into an area, where a system UID exists?

The only UIDs Buildah is going to map are defined in /etc/subuid and the UID running buildah if not root.


Simple Firm Investment Problem giving errors

using InfiniteOpt, Ipopt, Plots
δ=0.10; θ=2.00; r=0.15;
z= r + δ -.05;

T = 120.0   # horizon
k0 = 1.0 # capital endowment
u(i,k; z=z,θ=θ,δ=δ) = z*k -i -(θ/2)*k*(i/k - δ)^2 # dividend = return function
discount(t; r=r) = exp(-r*t) # discount function
BC(k, i; δ=δ) = i - δ*k        # LOM
opt = Ipopt.Optimizer   # desired solver
ns = 1_000;             # number of gridpoints

m = InfiniteModel(opt)
@infinite_parameter(m, t ∈ [0, T], num_supports = ns)
@variable(m, k, Infinite(t)) ## state variables
@variable(m, i, Infinite(t)) ## control variables

@objective(m, Max, integral(u(k,i), t, weight_func = discount))
ERROR: LoadError: /(::GeneralVariableRef,::GeneralVariableRef) is not defined. Are you trying to build a nonlinear problem? Make sure you use @NLconstraint/@NLobjective.
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:33
 [2] /(#unused#::GeneralVariableRef, #unused#::GeneralVariableRef)
   @ JuMP C:\Users\azevelev\.julia\packages\JuMP\b3hGi\src\operators.jl:418
 [3] u(k::GeneralVariableRef, i::GeneralVariableRef; z::Float64, θ::Float64, δ::Float64)
   @ Main c:\Users\azevelev\Dropbox\Computation\Julia\DE_OC_VFI\4prob\3a_Firm_Invest\Investment.jl:417
 [4] u(k::GeneralVariableRef, i::GeneralVariableRef)
   @ Main c:\Users\azevelev\Dropbox\Computation\Julia\DE_OC_VFI\4prob\3a_Firm_Invest\Investment.jl:417
 [5] macro expansion
   @ C:\Users\azevelev\.julia\packages\MutableArithmetics\8xkW3\src\rewrite.jl:279 [inlined]
 [6] macro expansion
   @ C:\Users\azevelev\.julia\packages\JuMP\b3hGi\src\macros.jl:1260 [inlined]
 [7] top-level scope
   @ c:\Users\azevelev\Dropbox\Computation\Julia\DE_OC_VFI\4prob\3a_Firm_Invest\Investment.jl:432
in expression starting at c:\Users\azevelev\Dropbox\Computation\Julia\DE_OC_VFI\4prob\3a_Firm_Invest\Investment.jl:432

Also, can this package solve infinite horizon problems yet?

What version of InfiniteOpt are you running? You'll need to update to the latest version for nonlinear expressions to work.
Infinite horizon problems are still on the todo list, I put most of my development time recently for adding general nonlinear support.  Admittedly, I don't foresee myself having much time in immediate future for adding full support for infinite-horizon problems, so we'll probably need someone else to lead that addition. We can currently represent infinite-horizon problems, we just need to add a transformation/solution approach to solve them in a general manner. I opened #190 to track this feature request.

Just updated. Still get errors:
using InfiniteOpt, Ipopt, Plots
p=1.00; δ=0.10; θ=2.00; r=0.15;
z= r + δ -.02
T = 120.0   # life horizon
k0 = 1.0 # endowment
u(k,i;z=z,θ=θ,δ=δ) = z*k -i -(θ/2)*k*(i/k - δ)^2 # utility function
discount(t; r=r) = exp(-r*t) # discount function
BC(k, i; δ=δ) = i - δ*k        # LOM

opt = Ipopt.Optimizer   # desired solver
ns = 1_000;             # number of gridpoints

m = InfiniteModel(opt)
@infinite_parameter(m, t ∈ [0, T], num_supports = ns)
@variable(m, k, Infinite(t)) ## state variables
@variable(m, i, Infinite(t)) ## control variables
@objective(m, Max, integral(u(k,i), t, weight_func = discount))
@constraint(m, c1, deriv(k, t) == BC(k, i; δ=δ))
@constraint(m, k == k0, DomainRestrictions(t => 0))
@constraint(m, k == 0, DomainRestrictions(t => T))

optimize!(m)

This is Ipopt version 3.14.4, running with linear solver MUMPS 5.4.1.

Number of nonzeros in equality constraint Jacobian...:     5999
Number of nonzeros in inequality constraint Jacobian.:        0
Number of nonzeros in Lagrangian Hessian.............:     3000

Number of Iterations....: 0

Number of objective function evaluations             = 0
Number of objective gradient evaluations             = 1
Number of equality constraint evaluations            = 0
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 1
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 0
Total seconds in IPOPT                               = 0.011

EXIT: Invalid number in NLP function or derivative detected.

Btw, I'm not sure if this is part of the issue, but there should be a salvage value at T, equal to e^(-r*T) * (1-\delta)K_T

It looks like the update resolved the nonlinear error and now InfiniteOpt is able to formulate the problem as you specify it.
Now Ipopt is throwing an error since the objective function is not well posed numerically since it divides by zero if k(t) has any points that approach 0. This is what occurs and induces the EXIT: Invalid number in NLP function or derivative detected. error from Ipopt. It appears that constraining k(t) to be strictly positive makes the problem infeasible. Adding the salvage value is a constant (it has not state/control variables in it), so adding it as an extra term to the objective wouldn't affect the solution.
Hence, the objective function will need to be reformulated into a better posed function. We can accomplish this by defining an auxiliary variable a == i / k - δ which we then pose as a quadratic constraint k * (a + δ) == i. Then we can use a when defining the objective function:
using InfiniteOpt, Ipopt
p = 1.00; δ = 0.10; θ = 2.00; r = 0.15
z = r + δ -.02
T = 120.0   # life horizon
k0 = 1.0 # endowment

m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t ∈ [0, T], num_supports = 1000)
@variable(m, k, Infinite(t)) ## state variables
@variable(m, i, Infinite(t)) ## control variables
@variable(m, a, Infinite(t))
@constraint(m, k * (a + δ) == i) # defines a == i / k - δ
@objective(m, Max, ∫(z * k - i - (θ / 2) * k * a^2, t, weight_func = t -> exp(-r * t)))
@constraint(m, c1, ∂(k, t) == i - δ * k)
@constraint(m, k(0) == k0)
@constraint(m, k(T) == 0)

optimize!(m)

This is able to then solve on my machine.
Since this doesn't constitute a bug in InfiniteOpt.jl, I please ask that further discussion on how to formulate this problem be done via the discussion forum (https://github.com/pulsipher/InfiniteOpt.jl/discussions).

This discussion is continued in #194.


Update JS integer lowering

Updates JS integer lowering to match the component model explainer (assuming that WebAssembly/component-model#52 is merged).
The main differences are that out-of-range numbers now wrap instead of throwing, and that non-integers are now rounded down instead of throwing.

Apologies for not merging this at the time I'm ot sure why I didn't earlier. My merge with main appears to have confused GitHub but I'll land this once CI passes


Start a network with a given set of epoch ledgers

task 2 from #6573

This would also mean we package epoch ledgers in the deb or make it available for downloading


Add depositor, deposit size and number of files to the collection items view

This view: https://sul-h2-stage.stanford.edu/collections/79
Should have all the columns as shown:
https://projects.invisionapp.com/share/EQXC9CLKCR2#/screens/420930435

Currently the collection show page is borrowing the dashboard collection component, but it should have it's own component: https://github.com/sul-dlss/happy-heron/blob/53050a549d2ac2d952799eb0f32d4c2319dee0dd/app/views/collections/show.html.erb#L33


Custom Origin

Hi, I'm using this example scatter graph:
https://github.com/chartjs/Chart.js/blob/master/samples/scatter.html
It works great, I was wondering if it is possible to move the axis origin (recentre the graph) from (0,0) to another value ej (20,40).
Thank you!

This is currently not possible.
Closing as duplicate of #2164


init satellite project

polish some ci tools && add a logger util

We should submit the go.mod file, I did not find the file now.

yes, I would merge this pr .


upgrade angular

Angular upgraded to v1.5.6.
PS: Built with JDK 1.8 and deployed in WildFly 10.0.0.Final

It's merged. Thank you very much for your contribution!
Cheers,


Double-check scraper code

edited to headless mode for unit tests in github actions, confirm everything still works as designed + pylint
also does she really need to scroll......... scrolling originally was for loading more jobs in linkedin - do we still need that functionality?

#136


Add --use-system-phantomjs flag

I'd like to be able to use spacejam, but I need to use phantomjs 2.0. This PR adds support for a --use-system-phantomjs flag that causes spacejam to not modify process.env.PATH before running phantomjs, so users that want to use their own build of phantomjs can do so.

Disregard my comments. All good. Can you please run cake test from the command line to see all tests pass for pure coffee too? In travis, I compile to js and only run the tests in pure node. Reason I didn't upgrade to phantomjs 2.0 is because native coffee support is discontinued, and I'm using the phantomjs coffee script when running cake coffee.

Yes, both cake test and npm test succeed from the command line.

Published as v1.2.1


/metrics does not respect --web.external-url configuration

Bug Report
What did you do?
I have configured --web.external-url=http://gajus.com/prometheus/.
What did you expect to see?
All Prometheus service endpoints to be prefixed with a path that is part of the --web.external-url.
What did you see instead? Under which circumstances?
The host is respected, but the path is not respected, i.e. Prometheus attempts to look for http://gajus.com/metrics instead of http://gajus.com/prometheus/metrics.
Environment

Prometheus version: v2.3.2
Prometheus configuration file:

https://github.com/helm/charts/tree/master/stable/prometheus +
alertmanager:
  persistentVolume:
    size: '32Gi'
server:
  baseURL: https://gajus.com/prometheus/
  prefixURL: /prometheus
storage:
  local:
    retention: '720h'

I have raised an issue with Helm charts too, just in case this is a configuration error.
https://github.com/helm/charts/issues/7088

Can you share your configuration file?
This setting configures the Prometheus http server, it has no effect on the scrape logic.

Can you share your configuration file?

Using the default configuration (https://github.com/helm/charts/blob/master/stable/prometheus/values.yaml) extended with the earlier mentioned configuration.

This setting configures the Prometheus http server, it has no effect on the scrape logic.

Point is that the http://gajus.com/prometheus/metrics endpoint exists and http://gajus.com/metrics does not. Since I have configured baseURL / prefixURL, I expect the checker to use the correct endpoint.
By the way, Prometheus does actually work as expected. I think it is just the liveness checker that is not aware of the prefix.

That sounds like an issue with the Helm chart, Prometheus is working as documented.

That sounds like an issue with the Helm chart, Prometheus is working as documented.

Which part of Prometheus is working as expected... ?

If by baseURL is https://gajus.com/prometheus/, then the above should query http://localhost:9090/prometheus/metrics or this assumption flawed?

Prometheus has no notion of baseURL, that's a Helm thing.

Prometheus has no notion of baseURL, that's a Helm thing.

--web.external-url, sorry.
https://github.com/helm/charts/blob/333bdfb086f70f4e8c3fe6f1c239418d6ff1e653/stable/prometheus/templates/server-deployment.yaml#L90-L92
Doesn't change the question.

Yes, Prometheus is working as expected. You told it to scrape /metrics so that's what it tried to do.

Yes, Prometheus is working as expected. You told it to scrape /metrics so that's what it tried to do.

Which part of configuration is that?

Which part of configuration is that?

I am not familiar with helm/charts at all, but if I were to guess, this is the place where another metrics path would have to be configured:
https://github.com/helm/charts/blob/4728544e0556cac4e52c70691f60c4ba31e36231/stable/prometheus/values.yaml#L789
No metrics_path is configured there, which means Prometheus takes the default of /metrics. This works as long as --web.external-url is not set. But as soon as it is modified, metrics_path would have to be adjusted (or rather: set at all) as well.
Relevant docs:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<scrape_config>
So you are probably right that this is an issue, but Prometheus-wise there is little which can be done as it is a configuration issue. It has therefore be fixed in the configuration files, which seem to be managed by helm/charts in your case. Therefore, It might make sense to open an issue in the helm/charts repository (maybe referencing this one).

I got it working after setting the correct metric_path . Here are my configurations:
apache virtualhost
 <VirtualHost *:443>
                ServerName servername.org
                ProxyPreserveHost on
                ProxyPass /prom/ http://localhost:9090/prom/
                ProxyPassReverse /prom/ http://localhost:9090/prom/
                <Location /prom>
                    AuthType Basic
                    AuthName "Restricted Content"
                    AuthUserFile /etc/apache2/.htpasswd
                    Require valid-user
                </Location>
                SSLEngine on
                SSLCertificateFile    /etc/letsencrypt/live/servername.org/cert.pem
                SSLCertificateKeyFile /etc/letsencrypt/live/servername.org/privkey.pem
                SSLCertificateChainFile /etc/letsencrypt/live/servername.org/chain.pem
</VirtualHost>

prometheus.yml
global:
  scrape_interval:     10s
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets:
        - 'localhost:9090'
    metrics_path: '/prom/metrics/'

prometheus.service (systemd)
[Unit]
Description=Prometheus Server

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file=/etc/prometheus/prometheus.yml \
    --storage.tsdb.path=/var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries \
    --web.external-url=https://servername.org/prom/
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
Restart=always

[Install]
WantedBy=multi-user.target

Now I can get the prometheus explorer at https://servername.org/prom/ (the tailing slash is important here)