DynmapForge not working with LuckPerms

I have installed SpongeForgeAPI 5,  LuckPerms, and DynmapForge on a MC 1.10.2 server. Unfortunately, not even a group or user set with all (*) permissions can use Dynmap commands as the user is told that they don't have permissions to use them.

Use the verbose command to check which permission is required by the commands.

It is calling for "dynmap.command.dmarker" and "dynmap.command.dynmap" for the commands that I want to use, but I am part of a group that has permission * set to true. Isn't that supposed to take care of that? I haven't had any problems with any other permissions with exception to this on.

I just used "/lp user set  permission set dynmap.command.dynmap true" and "/lp ... dynmap.command.dmarker true" also, just to see if it would work, but it did not.

probably the forge mod lacks such permission support.

Again, please can you check with /lp verbose to see if that node is actually being checked for.

Okay. I suppose that I should have included a screenshot with my last entry, as "it is calling for" was referring to /lp verbose. This time I have included a screenshot for clarity.

Ok yeah, since those logs are showing a result for TRUE is being returned, then this is definitely a mod issue. :/

Why was this closed? I got exactly the same problem. So my only option is to move over to a different permission plugin?

Cause it's mod issue?

And that means what?

Dynmap

OK thanks.

FWIW @YorVeX , Dynmap-3.0-alpha-1-forge-1.12.2 and LuckPerms-Sponge-4.1.30 work perfectly with each other on Sponge Forge API 7.1.0-BETA-2958 and Forge 1.12.2-<IP_ADDRESS>15.  So it doesn't surprise me that 1.10.2 is having issues. :-)  You may find an updated version of Dynmap that may fix your issue for 1.10.2.

@smmmadden I am already on 1.12.2 but I was using Dynmap-2.6-beta-1-forge-1.12.jar so far, didn't even know that a 3.0 alpha existed. Was checking before posting here whether there is a newer version and didn't find this one. Will try the combination of versions you have mentioned now.

I didn't find LuckPerms-Sponge-4.1.30 for download anywhere. I the one I had (43) for testing and tried both the other versions you mentioned and newer versions of these components. In detail I tried these two combinations:

Dynmap-3.0-alpha-1-forge-1.12.2 + LuckPerms-Sponge-4.1.43 + Sponge Forge API 7.1.0-BETA-2958 + Forge 1.12.2-<IP_ADDRESS>15
Dynmap-3.0-alpha-1-forge-1.12.2 + LuckPerms-Sponge-4.1.43 + Sponge Forge API 7.1.0-BETA-2970 + Forge 1.12.2-<IP_ADDRESS>35

Result is the same for both combinations:

Maybe when you tried you were op on your server? Because it works when I am op. But I want to give that permission to someone without having to give him op. Or it's the older LP version that you are using which is making the difference (which I doubt).

This is a Dynmap issue and has nothing to do with LuckPerms.
DynmapForge supports two ways of checking for permissions. You can choose between using the file based internal permission system, or just checking if the user is op.
https://github.com/webbukkit/DynmapForge/tree/master/src/main/java/org/dynmap/forge/permissions
There's nothing I can do about that. You need to ask the author of Dynmap to add support for proper permission checks.

Posted an issue there. In the meantime I am using the file based permissions, it's working now. For anyone else finding this post and wondering how they can set marker permissions in DynmapForge I posted an example of a permissions.yml file here that has to be placed on the /dynmap folder on the server: https://pastebin.com/M1VADY7u

3 variables that are different in my scenario.

I am an admin
I am OP level 4
I do NOT allow any dynmap commands for players. The map is provided as a courtesy, not as a feature that can be manipulated.
So if the dmarker perms do not work in the release you are testing with, report it to the Dynmap Team.

@smmmadden If your case is different, you should rather open a new issue.

i'm not reporting any issue, merely replying to @YorVeX - everything is working fine for my setup.


I wanted to comment on HN but they already flagged you

Doing it here:

You smartarse :)

Haha, someone posted this on HN?
I still love Rails if that counts for anything. 😃


Use LOGIN_REDIRECT_URL setting in login view.

Proposed fix for issue #3.

Alright, I'll make success_redirect default to None, then use LOGIN_REDIRECT_URL and then default to the previous default of /account/ so as to not break existing code.


Can flowtype automatically annotate the code?

For example:
function add(x){return x + 10;}
can be changed to:
function add(x:string){return x + 10;}
or
/*@params {string} x*/ function add(x:string){return x + 10;}

How would Flow know you didn't mean function add(x: number){return x + 10;}?
In general, no, Flow cannot annotate your code perfectly, but you can try flow suggest <file>. It should work well for output types but not so well for input types (as in your example).

thx a lot!


Pkg.build(BioMedQuery) error on Julia 0.6.4

Hello,
I am having an issue when I attempting to build BioMedQuery on Julia 0.6.4 and get the following build error with one of BioMedQuery's dependencies SQLite. I am running the Julia on windows 10 machine.
`julia> Pkg.build("BioMedQuery")
INFO: Building CodecZlib
INFO: Building LibCURL
INFO: Building LightXML
INFO: Building MbedTLS
INFO: Building SpecialFunctions
INFO: Building DecFP
INFO: Building MySQL
INFO: Building Gumbo
INFO: Building WinRPM
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_Leap_42.2/repodata/repomd.xml
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_Leap_42.2/repodata/repomd.xml
INFO: Building SQLite
INFO: Updating WinRPM package list
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_Leap_42.2/repodata/repomd.xml
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_Leap_42.2/repodata/repomd.xml
INFO: Nothing to do
===================================================[ ERROR: SQLite ]====================================================
LoadError: Provider BinDeps.PackageManager failed to satisfy dependency sqlite3_lib
while loading C:\Users\Thor.julia\v0.6\SQLite\deps\build.jl, in expression starting on line 19
========================================================================================================================
====================================================[ BUILD ERRORS ]====================================================
WARNING: SQLite had build errors.
packages with build errors remain installed in C:\Users\Thor.julia\v0.6
build the package(s) and all dependencies with Pkg.build("SQLite")
build a single package by running its deps/build.jl script
julia>`

Hi @anthonyagudelox3, thanks for submitting this issue.
Unfortunately, Julia 0.6 is no longer supported. I would recommend that you transition to Julia 0.7 and then subsequently to Julia 1.0.
If you encounter an error building BioMedQuery.jl on Julia 0.7 and/or Julia 1.0, please open a new issue.


Use the curl_reset function to allow resource reusing

Since the PHP required version is 5.5, we can rely on the curl_reset function to manage curl handle.
It allow us to avoid create a new resource for each call. I've faced to some problem without reusing handles on previous project. Since that I always use that. It just reset all the attached option to the current handle.

could there be an issue with async requests? or can curl never do async anyways?

Actually we could backport curl_reset to PHP 5.4.
Is that possible at all? Maybe we could check if there is a symfony pollyfill for it.

It's not possible to create a polyfill to curl_reset. I've already tried to find a way to do that but we can't unset a config, we can just set its value.
I think that this PR is not a problem about the async thing, we just need to ensure that all is fine when async implementation is done.

i would not even try to support php 5.4. its end of life. see also #2

Yeah, if there is 1% need for PHP 5.5, use it.


🐛 [BUG] - deleting a file actually deletes all files

Description
everything is in the title.
Must only delete the selected file
Reproduction URL
https://github.com/ThomasSed/Storz
Reproduction steps
1. Go to repository
2. Click on a file
3. Click on the trash button
4. accept
5. Go back to repository

Screenshots
![DESCRIPTION](LINK.png)

Logs
No response
Browsers
No response
OS
No response

Hi @ThomasSed,
This is happening because both files(having the same file names) have the same CID.
Currently, we are querying files for deletion based on their CID. If we can prevent this #104, we might be able to solve this too.

a way to do this is by hashing the file using HMAC, and not simply sha-256.
storing the generated secret key on the db must do the trick here.
example (https://github.com/brix/crypto-js) :
import hmacSHA512 from 'crypto-js/hmac-sha512';
import Base64 from 'crypto-js/enc-base64';
const  privateKey = uuid();
const hmac = Base64.stringify(hmacSHA512(file, privateKey));


Date picker in the forms

If a property is of type date, have the form have a date picker and use the property ._i to get back the iso string: YYYY-MM-DD

Created as task derived from baobabsoluciones/cornflow-app#47


[Bug]: Issue occurs in setuptools version 70 +

Describe the bug
https://github.com/openvinotoolkit/anomalib/blob/41d8cf90cfb8de954fd3ed0ec9c2c7bff75b81d3/src/anomalib/models/video/ai_vad/clip/clip.py#L21
An error occurs in environment where setuptools is version 70 or higher. This is related to this issue. https://github.com/pypa/setuptools/issues/4376
This can cause issues with newer builds.
Dataset
N/A
Model
N/A
Steps to reproduce the behavior
Install anomalib==1.1.0 with setuptools==70.0.0
pip install setuptools==70.0.0
pip install 'anomalib[core]==1.1.0'
anomalib train ***

OS information
OS information:

OS: [e.g. Ubuntu 20.04]
Python version: [e.g. 3.10.0]
Anomalib version: [e.g. 0.3.6]
PyTorch version: [e.g. 1.9.0]
CUDA/cuDNN version: [e.g. 11.1]
GPU models and configuration: [e.g. 2x GeForce RTX 3090]
Any other relevant information: [e.g. I'm using a custom dataset]

Expected behavior
Install without error
Screenshots
Problem with given class_path 'otx.algo.anomaly.padim.Padim':
cannot import name 'packaging' from 'pkg_resources' (/mnt/hdd1/miniforge3/envs/py310-otx/lib/python3.10/site-packages/pkg_resources/__init__.py)

Pip/GitHub
pip
What version/branch did you use?
1.1.0
Configuration YAML
N/A

Logs
Problem with given class_path 'otx.algo.anomaly.padim.Padim':
cannot import name 'packaging' from 'pkg_resources' (/mnt/hdd1/miniforge3/envs/py310-otx/lib/python3.10/site-packages/pkg_resources/__init__.py)

Code of Conduct

[X] I agree to follow this project's Code of Conduct

https://github.com/openvinotoolkit/anomalib/pull/2082 should solve this issue when it is merged.


XGBoost External Memory Version Load SVM File Into DMatrix Error

import os
from typing import List, Callable
import xgboost
from sklearn.datasets import load_svmlight_file

class Iterator(xgboost.DataIter):
  def __init__(self, svm_file_paths: List[str]):
    self._file_paths = svm_file_paths
    self._it = 0
    # XGBoost will generate some cache files under current directory with the prefix
    # "cache"
    super().__init__(cache_prefix=os.path.join(".", "cache"))

def next(self, input_data: Callable):
    """Advance the iterator by 1 step and pass the data to XGBoost.  This function is
    called by XGBoost during the construction of ``DMatrix``

    """
    if self._it == len(self._file_paths):
      # return 0 to let XGBoost know this is the end of iteration
      return 0

    # input_data is a function passed in by XGBoost who has the exact same signature of
    # ``DMatrix``
    X, y = load_svmlight_file(self._file_paths[self._it])
    input_data(X, y)
    self._it += 1
    # Return 1 to let XGBoost know we haven't seen all the files yet.
    return it

  def reset(self):
    """Reset the iterator to its beginning"""
    self._it = 0

it = Iterator(["my.svm"])
Xy = xgboost.DMatrix(it)

my.svm file:
1 0:0.45 2:0.45 3:0.89
0 0:0.45 2:0.45 3:0.89
1 0:0.45 2:0.45 3:0.89

error message:
ValueError: feature_names must be unique
xgboost.__version__
1.6.1

Please use keyword arguments for the input_data.


Typo

@sveinrune : Thanks for your contribution to the Azure documentation! The author, @jimdial, has been notified to review your proposed change.

@sveinrune Thanks for catching! #sign-off


Fix for explicit Image pane sizing

Codecov Report

Merging #43 into master will decrease coverage by 0.22%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master      #43      +/-   ##
==========================================
- Coverage   92.21%   91.98%   -0.23%
==========================================
  Files          21       21
  Lines        2504     2509       +5
==========================================
- Hits         2309     2308       -1
- Misses        195      201       +6

Impacted Files
Coverage Δ

panel/pane.py
80.19% <50%> (-1.66%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef75310...d6f6ad9. Read the comment docs.

Codecov Report

Merging #43 into master will decrease coverage by 0.22%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master      #43      +/-   ##
==========================================
- Coverage   92.21%   91.98%   -0.23%
==========================================
  Files          21       21
  Lines        2504     2509       +5
==========================================
- Hits         2309     2308       -1
- Misses        195      201       +6

Impacted Files
Coverage Δ

panel/pane.py
80.19% <50%> (-1.66%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef75310...d6f6ad9. Read the comment docs.

That fixes issues for me; thanks!


[BUG] It seems like there's a memory leak

Aged Version: 3.0.2

My memory usage keeps increasing even when i limit jvm memory usage to just 4gigs. It started to ballon to 6 gigs fill up my memory, and when my ram is full, it then started to fill up my swap as well. Here's a screenshoot of memory usage

Could be cause of Spyglass Improvements.
Pls reopen if this still happens in v3.1.1 or newer


Flaky test suite after migration from 9.1.0 to 10.2.0

Environment
Elixir: v1.15.7
otp: 26.1.2
Sentry 10.2.0
Steps to Reproduce
When migrating from 9.1.0 and put this config on my config/test.exs file :
config :sentry, test_mode: true, dsn: nil

Expected Result
The suite would pass as normally.
Actual Result
Flaky errors like this one
     ** (ArgumentError) errors were found at the given arguments:

       * 1st argument: the table identifier does not refer to an existing ETS table

     code: assert {{code}}
     stacktrace:
       (stdlib 5.0.2) :ets.update_element(Sentry.Dedupe, 14786931, {2,<PHONE_NUMBER>364})
       (sentry 10.2.0) lib/sentry/dedupe.ex:23: Sentry.Dedupe.insert/1
       (sentry 10.2.0) lib/sentry/client.ex:158: Sentry.Client.maybe_dedupe/1
       (sentry 10.2.0) lib/sentry/client.ex:123: Sentry.Client.send_event/2

Then I tried this config:
config :sentry, test_mode: true, dsn: nil, dedup_events: false, send_result: :none

And everything seems to be more stable. No flaky errors.
Is this how is supposes to be?

@ulissesalmeida hey Ulisses, long time no chat! 🙃
Are you starting the :sentry app in the test Mix env?

Is this how is supposes to be?

Nope 😬

@whatyouhide Long time ❤  I have been hiding below my desk for a while 🤣

Are you starting the :sentry app in the test Mix env?

Thank you for asking that. I thought not. But, after investigating the code, I noticed that we're stopping and starting the sentry app for changing the configuration to test with Bypass. 😅  I'll fix that, it is supposed to use the Config.change function right? After that, I'll come back  here to tell if this weird behaviour is still happening.

@ulissesalmeida yes you should use Sentry.Config.put_config/2 now. However, there's a whole new Sentry.Test module in 10.2.0 you might want to look at 😉

@whatyouhide Using the put config function made our test suite be stable(we're no stopping or starting the app anymore).
We'll soon move the current tests to use the new test module, the new module looks gorgeus.
Thank you for your quick reply and help.
❤ 💛 💜 💚


Dotnet run command fail

Problem encountered on https://dotnet.microsoft.com/en-us/learn/dotnet/hello-world-tutorial/run
Operating System: macos
/usr/local/share/dotnet/sdk/7.0.402/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(594,5): error NETSDK1177: Failed to sign apphost with error code 0: xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun [/Users/anathi/MyApp/MyApp.csproj]
/usr/local/share/dotnet/sdk/7.0.402/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(594,5): error NETSDK1177: /Users/anathi/MyApp/obj/Debug/net7.0/apphost: the codesign_allocate helper tool cannot be found or used [/Users/anathi/MyApp/MyApp.csproj]
/usr/local/share/dotnet/sdk/7.0.402/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(594,5): error NETSDK1177:  [/Users/anathi/MyApp/MyApp.csproj]
The build failed. Fix the build errors and run again.

Hello,
Looks like some of the necessary tools don't exist on your machine. Can you try to run xcode-select --install first and then can you run dotnet run again?
cc: @mairaw

Thank you @liveans, your solution worked :)


Support for hierarchical plans

This creates a HierarchicalPlan class that contains:

a non-hierarchical plan (either TimeTriggeredPlan or SequentialPlan)
decomposition information that maps an initial task network down to primitives actions

@alvalentini This is ready for review now. We have used internally for some time without any problem.
We intend to submit a plan validator that leverages it quickly. It might also be useful for the integration of Siadex from the open calls.


Support for multiple Bluetti devices

Hi,
Is there a way to collect Data of 2 or more battery packs with the script?

Do you mean multiple power stations or multiple batteries connected to a single power station. Currently you can monitor multiple power stations simultaneously by simply providing additional bluetooth addresses to the script as arguments.

Yes, i meant multiple power stations. Thanks for the reply

I tried it with the newest version of your script and it worked. The only misbehavior i noticed is that the measurements getting very irregularly despite of the --interval parameter


TensorRT Optimization

Is there any TensorRT optimised inference available?

No

StyleTTS2 consists of several models and components that work together to generate audio. To optimize it using TensorRT, you first need to convert each model separately from PyTorch to ONNX. Once converted, you can either:
Run the ONNX models using ONNX Runtime with the TensorRT execution provider, or
Convert the ONNX models directly into TensorRT format and perform inference using TensorRT's Python or C++ API.
From my experience with ablation studies, the decoder is the most resource-intensive component in StyleTTS2. If you aim for partial optimization, converting just the decoder from PyTorch to ONNX and running it in TensorRT can provide significant speed improvements. Alternatively, converting all models to ONNX and running them in ONNX Runtime with the TensorRT execution provider will also yield noticeable performance gains. This approach is feasible and I have tested it.

HI @UmerrAhsan. Could you please share the overall latencies of your ONNX model?

Hi @nityanandmathur. I have ran the decoder model and predictor.text encoder model in tensorrt. It decreases my latency by over 50%. Also I have cached the style vectors from diffusion and style encoder before.


Add metrics to SimpleScheduler and Worker

Adds metrics for running tasks and such to prometheus.

This change is

cc: @chrisstaite-menlo, this should give you the metrics you were wanting.

cas/scheduler/default_scheduler_factory.rs line 105 at r1 (raw file):
Previously, allada (Nathan (Blaise) Bruer) wrote…
Yeah, I did consider that. The major reason I liked doing it this way is because it is more standard to how large projects manage it. Because there's a decent chance in the future we may make unregister_metrics() to allow temporary attachment of metrics for debugging. It's more about long-term flexibility.

Is it worth complexity now for possible future expansion?  I don't mind either way, and it's only during start-up, so it's not like it's a major issue.


[proposal] Remove bootstrap as bower dependency

Are there any dependencies from Bootstrap other than this:
.dropdown-menu {
	position: absolute;
	top: 100%;
	left: 0;
	z-index: 1000;
	display: none;
	min-width: 160px;
	padding: 5px 0;
	margin: 2px 0 0;
	font-size: 14px;
	text-align: left;
	list-style: none;
	background-color: #fff;
	background-clip: padding-box;
	border: 1px solid #ccc;
	border: 1px solid rgba(0,0,0,.15);
	border-radius: 4px;
	box-shadow: 0 6px 12px rgba(0,0,0,.175);
}

I use it without Bootstrap (only added your CSS, your JS and the CSS I posted here) in a project and I think it's unnecessary to list bootstrap as dependency.

On v 3.0.0 there will be a big refactoring that will make the plugin optionally independent from bootstrap


CI: run plotjuggler test on mac

Added this because I was hitting that pycurl issue again, but looks like it works in CI. @fredyshox any idea why it's happening?
macbookpro:plotjuggler adeeb$ ./juggle.py
Traceback (most recent call last):
  File "/Users/adeeb/openpilot/tools/plotjuggler/./juggle.py", line 15, in <module>
    from tools.lib.logreader import LogReader
  File "/Users/adeeb/openpilot/tools/lib/logreader.py", line 11, in <module>
    from tools.lib.filereader import FileReader
  File "/Users/adeeb/openpilot/tools/lib/filereader.py", line 2, in <module>
    from tools.lib.url_file import URLFile
  File "/Users/adeeb/openpilot/tools/lib/url_file.py", line 8, in <module>
    import pycurl
ImportError: pycurl: libcurl link-time version (7.85.0) is older than compile-time version (7.87.0)


refacotring proposal for aggregates management

Differentiate Aggregate State from Aggregate behavior
Make very clear that a Contribution is both a State and an Aggregate (see type State = Self;)
Pass state as value in apply() so it cannot be used anymore and forces to be event sourced again

:recycle: make Aggregate state immutable
:recycle: pass events as reference in aggregate
:recycle: remove event application from aggregate command as per SRP

Codecov Report

Merging #193 (a3aa7c5) into main (7d866f8) will increase coverage by 0.14%.
The diff coverage is 96.87%.

@@            Coverage Diff             @@
##             main     #193      +/-   ##
==========================================
+ Coverage   73.89%   74.03%   +0.14%
==========================================
  Files         124      125       +1
  Lines        3869     3867       -2
==========================================
+ Hits         2859     2863       +4
+ Misses       1010     1004       -6

Impacted Files
Coverage Δ

...etplace-core/src/application/contribution/apply.rs
0.00% <0.00%> (ø)

marketplace-domain/src/aggregate.rs
100.00% <100.00%> (+20.00%)
:arrow_up:

...arketplace-domain/src/aggregate_root_repository.rs
97.77% <100.00%> (ø)

...lace-domain/src/contribution/aggregate_root/mod.rs
95.83% <100.00%> (-3.19%)
:arrow_down:

...ace-domain/src/contribution/aggregate_root/test.rs
100.00% <100.00%> (ø)

marketplace-infrastructure/src/database/schema.rs
0.00% <0.00%> (ø)

...etplace-infrastructure/src/database/event_store.rs
95.12% <0.00%> (+0.25%)
:arrow_up:

marketplace-core/src/routes/dto/api_key.rs
39.13% <0.00%> (+8.09%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Inconsistent entries in /etc/passwd file

Our /etc/passwd file contains some inconsistency:
root@830652dcfff3:~# pwck
user 'games': directory '/usr/games' does not exist
user 'man': directory '/var/cache/man' does not exist
user 'lp': directory '/var/spool/lpd' does not exist
user 'news': directory '/var/spool/news' does not exist
user 'uucp': directory '/var/spool/uucp' does not exist
user 'www-data': directory '/var/www' does not exist
user 'list': directory '/var/list' does not exist
user 'irc': directory '/run/ircd' does not exist
user 'uuidd': directory '/run/uuidd' does not exist
pwck: no changes

We need to remove the error messages for our testing. We can either remove this by deleting the users or changing the entries.
Removing the entries changes the base assumption many packages might have, like nginx.
oot@830652dcfff3:~# apt-get install nginx
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  nginx-common
Suggested packages:
  fcgiwrap nginx-doc ssl-cert
The following NEW packages will be installed:
  nginx nginx-common
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 640 kB of archives.
After this operation, 1696 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 https://cdn-aws.deb.debian.org/debian bookworm/main amd64 nginx-common all 1.22.1-9 [112 kB]
Get:2 https://cdn-aws.deb.debian.org/debian bookworm/main amd64 nginx amd64 1.22.1-9 [527 kB]
Fetched 640 kB in 0s (1345 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package nginx-common.
(Reading database ... 20513 files and directories currently installed.)
Preparing to unpack .../nginx-common_1.22.1-9_all.deb ...
Unpacking nginx-common (1.22.1-9) ...
Selecting previously unselected package nginx.
Preparing to unpack .../nginx_1.22.1-9_amd64.deb ...
Unpacking nginx (1.22.1-9) ...
Setting up nginx-common (1.22.1-9) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.)
debconf: falling back to frontend: Readline
chown: invalid user: 'www-data:adm'
dpkg: error processing package nginx-common (--configure):
 installed nginx-common package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of nginx:
 nginx depends on nginx-common (<< 1.22.1-9.1~); however:
  Package nginx-common is not configured yet.
 nginx depends on nginx-common (>= 1.22.1-9); however:
  Package nginx-common is not configured yet.

dpkg: error processing package nginx (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 nginx-common
 nginx
E: Sub-process /usr/bin/dpkg returned an error code (1)

Instead, I'm opting to change the default path somewhere else.

My solution for now is to remove all but www-data.


prepare for 0.6.2 release

Changeset since 0.6.1
Includes:

#129
#132
#133
#135

Looks like the build succeeded but the notification back to GitHub got lost somewhere...


Phase 5 - Rewrite Interaction Manager

Externs for the new Interaction Manager classes
https://github.com/GoodBoyDigital/pixi.js/tree/dev-rewrite/src/interaction

Done.


Babel Optimization Error: Module Size Exceeding 500KB

Description
When building a SolidJS project using solid-icons, Babel throws an optimization error indicating that the styling of some modules exceeds the maximum size of 500KB. This is occurring specifically with solid-icons/fa/index.js and solid-icons/tb/index.js. The error message is repeated multiple times, suggesting that the issue persists through multiple attempts to compile.
Steps to Reproduce

Set up a SolidJS project with @solidjs/start.
Include the solid-icons package in the dependencies.
Run the build script.

Expected Behavior
The build should complete without errors, and Babel should handle the module size appropriately, possibly by splitting or tree-shaking.
Actual Behavior
The following error is received:
[BABEL] Note: The code generator has deoptimised the styling of /home/don/MEGA/Projects/coding.global-web/node_modules/solid-icons/tb/index.js as it exceeds the max of 500KB.
[BABEL] Note: The code generator has deoptimised the styling of /home/don/MEGA/Projects/coding.global-web/node_modules/solid-icons/fa/index.js as it exceeds the max of 500KB.

Environment

SolidJS Version: 1.8.15
Solid Icons Version: 1.1.0
Node Version: 20
Package Manager: [yarn]
Operating System: [Linux]

Possible Solution
Could this be an issue with how the icons are being imported or bundled? Is there a recommended way to import solid-icons that could avoid this problem?
Additional Context

Hi @0-don thanks for sharing, solid-start receive new updates recently, i will check this and go back with update soon.

In dev mode, Babel shows the error as above. However, when I built it, app works well. I think 'tree shake' function solved this issue during the build process. Am I right?

Just noticed the same issue here.

same hare

The one fa index file is over 2k lines and they all look like this:
export declare const FaSolidY: IconTypes;
export declare const FaSolidYenSign: IconTypes;
export declare const FaSolidYinYang: IconTypes;
export declare const FaSolidZ: IconTypes;

There is no way to import one of these.  Perhaps all these icons need separate files, but that is a pretty rough way to fix the issue.  I think the code generator is going to need to be a bit smarter.

I agree, in fact I've been iterating recently on how to make icon generation more optimized, going for files separated by icons and their types sounds the most reasonable, I remember that I decided to export the types of each icon name in that way because that way VSCode suggested the names automatically.
It's possible that the reason for doing it that way is that it's not logical for the library's operation.


Settings > Tokens: Site Area must be hidden when organization is inactive (column in list + input field in pop-up)

This works as intended. Closing issue.

This works as intended. Closing issue.


TAXII connector not showing

There is not Threat Intelligence - TAXII (Preview) connector showing in the list.  Is this a private preview offering?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0588fd19-837a-95f8-b300-558d385056f3
Version Independent ID: c51f135d-8378-b263-aed8-3de8e8f593ed
Content: Connect threat intelligence data to Azure Sentinel
Content Source: articles/sentinel/connect-threat-intelligence.md
Service: security-center
GitHub Login: @cabailey
Microsoft Alias: cabailey

@garybushey I have received the information that this is still in private preview and it should be in public preview in the coming weeks.

@garybushey We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Adding extra Cassandra configuration parameters

It fails on
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project storm: Too many files with unapproved license: 3 See RAT report in: /home/travis/build/apache/storm/target/rat.txt -> [Help 1]

Do I need to do anything with licenses?

@tandrup can you please a file a JIRA https://issues.apache.org/jira/browse/ under STORM project. Also update the title of the JIRA and squash the commits in this PR. More details https://github.com/apache/storm/blob/master/DEVELOPER.md#contribute-code
https://github.com/apache/storm/blob/master/DEVELOPER.md#pull-requests

@harshach OK. I was hoping my changes was small enough to fall into the small patches category. I'll create a jira.

@tandrup for small docs changes etc. we don't file JIRAs but this one had quite few changes good to have that in JIRA and subsequently in CHANGELOG.

overall LGTM. +1 .
@tandrup  would like to see these configs documented here https://github.com/apache/storm/blob/master/external/storm-cassandra/README.md

@harshach I've added the new configs to the existing README list.

+1


Smart Source Freshness Runs

resolves #4050

Description

When a user wants to run dbt models based on freshest data possible, they can do so in an elegant way: dbt build --select source_fresh:yes+ --defer --state <previous state path>
Example Use Case
dbt Cloud Job A: runs source freshness every 30 minutes. It will store 2 versions of the sources.json freshness output. Each invocation will replace BOTH the current state and the previous state sources.json.
current state: target/sources.json
previous state: target/archive_sources/sources.json
Command Steps:

dbt source freshness

dbt Cloud Job B: runs every 35 minutes and defers to Job A above. If the max_loaded_at date for the current state source freshness check is greater than the max_loaded_at for the previous state, this command will run the downstream models in scope. This does NOT run the dbt source freshness command as all of that is happening within Job A. The job should only have to look within the ./target folder of Job A. Using the build command tests the sources for quality, so if those checks fail, downstream nodes will not run.
Command Steps:

dbt build --select source_fresh:yes+ --defer --state <previous state path>

Considerations:

Will dbt Cloud have to add new functionality to store previous state given that dbt command steps are stateless by default?

Checklist

[x] I have signed the CLA
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] I have updated the CHANGELOG.md and added information about my change

@sungchun12 @anaisvaillant Very cool, very quick work!
I took a quick look, and I noticed a difference between the implementation here and the idea described in #4050, though please tell me if I'm missing something.
The proposal in #4050 is to compare max_loaded_at between <state_path>/sources.json (previous run) and <target_path>/sources.json, to figure out if the "bookmark" for that source has moved. Two distinctions:

We need to read in two different sources.json files. I think that might look like registering both PreviousState.sources (using PreviousState.path), and also something like CurrentState.sources (using target_path). The latter isn't something that exists just yet; we might need to think a bit about where it ought to be defined and pull in.
I think we want the source's max_loaded_at, rather than its status. It could be fresh one or both times, without having moved the bookmark.

Once we have a way to register both, and perform the comparison between their max_loaded_at, I envision something like (pseudo-code):
class SourceBookmarkSelectorMethod(SelectorMethod):
    def search(
        self, included_nodes: Set[UniqueId], selector: str
    ) -> Iterator[UniqueId]:
        if self.previous_state is None or self.previous_state.sources is None
        or self.current_state is None or self.current_state.sources is None:
            raise InternalException(
                'Comparison sources missing in {self.previous_state.path} or {self.current_state.path}
            )
        # select any "current" source (just freshness-checked) that has a greater max_loaded_at
        # than the same source (previously freshness-checked), or if the source did not previously exist
        previous_sources = self.previous_state.sources
        current_sources = self.current_state.sources
        matches = set(
            current_source.unique_id for current_source in current_sources
            if (
                current_source.unique_id not in previous_sources or
                current_source.max_loaded_at > previous_sources[current_source.unique_id].max_loaded_at
            )
        )
        for node, real_node in self.all_nodes(included_nodes):
            if node in matches:
                yield node

@jtcohen6 You raise a solid point that a source_status:pass+ without a max_loaded_at bookmark quickly becomes a wasteful run of children nodes if it's run every 5 minutes. I'll figure out where to inject CurrentState.sources.
Options on selector syntax and mechanics: @jtcohen6, Do you agree with my recommendation?

dbt run --select source_status_max_loaded_at:pass+ This provides explicit intuition for the developer upfront that source status matters, and it'll only run children nodes if the max loaded at date has a date greater than the previous source freshness date. I still see use cases where warn and error statuses are used to run children nodes rather than max_loaded_at alone.  However, it's a long name for a selector flag. Sung's Recommendation

dbt run --select source_status:pass+. This implies BOTH the source status AND the max loaded at date are explicit considerations before children nodes are run. Still the same flag. However, a developer will have to look at documentation to understand that both mechanisms are at play.

Two separate selectors that include source_status:pass+ AND source_status_max_loaded_at: pass+. This way, the developer can choose their own mechanism to run children nodes on status criteria OR status and max loaded at criteria.

I want to avoid using sources:fresh+ because it requires the developer to research on their own what mechanisms are used to run children nodes.

@sungchun12 I think I agree with your first recommendation. I want to hear a bit more about:

This provides explicit intuition for the developer upfront that source status matters... I still see use cases where warn and error statuses are used to run children nodes rather than max_loaded_at alone.

While source freshness statuses (pass, warn, error) matter tremendously for team SLAs and downstream metadata (e.g. exposure status reporting), I don't see how I'd want to use them to inform selection criteria. From my point of view, I just want the comparison between old + new max_loaded_at values. Does source X have new data since the last time I ran my models? If so, I want to select models downstream of source X to re-run; if not, I don't.
I feel that holds true regardless of whether the old max_loaded_at was past the warn threshold, the new max_loaded_at is past the error threshold, or both are safely within my freshness expectations and pass. In all cases, I just want to know whether the bookmark has moved.

@jtcohen6 Thanks for the thorough reply! I also updated the selector flag to be shorter and coherent source_refresh:pass+. I'll provide examples below of how this feature will be used in the wild in response to your point on max_loaded_at criteria.
I agree that previous state pass, warn, error thresholds are irrelevant for source refresh jobs. I still recommend pass, warn, error thresholds are relevant for current state.
Open Questions

Do you agree with this recommendation?
When does this feature need to be merged to make it to the first release of v1.0.0?

Previous State dbt job
$ dbt source freshness # this job only checks data freshness

Current State dbt job
$ dbt source freshness # run this again to compare `max_loaded_at` to the previous state
$ dbt build -s source_refresh:pass+ source_refresh:warn+ --state my/previous/run/artifacts # refresh children nodes of sources that have `pass` or `warn` AND have fresher `max_loaded_at` dates ONLY in the current state dbt job

Today, a dbt developer has an intuition that children nodes of sources should run based on pass, warn, error. For the majority, pass, warn are the driving criteria to run children nodes. If this feature for source refreshes removes that criteria and only considers max_loaded_at, it takes away that driving criteria. Including the pass, warn criteria as concurrent selectors gives the developer more choices to mix and match use cases.
I don't see a compelling use case where a source refresh run should do the below, assuming all sources have error.
Previous State dbt job
$ dbt source freshness # this job only checks data freshness

Current State dbt job
$ dbt source freshness # run this again to compare `max_loaded_at` to the previous state. All sources have fresher `max_loaded_at` dates, but all are `error`
$ dbt build -s sources:fresh+ --state my/previous/run/artifacts # refresh children nodes of sources that have fresher `max_loaded_at` dates even if they are all `error`

Developers are expecting source refresh runs to be fresh AND meeting thresholds(ex: pass, warn) not only fresh.
I anticipate developers getting confused on why children nodes are being run if all sources have freshness errors. I also imagine lots of downstream tests failing because data isn't meeting the source pass criteria. However, if you have something like the below, this gives clarity to the logs and troubleshooting process.
$ dbt source freshness # run this again to compare `max_loaded_at` to the previous state. All sources have fresher `max_loaded_at` dates, but all are `error`
$ dbt build -s source_refresh:error+ --state my/previous/run/artifacts # refresh children nodes of sources that have fresher `max_loaded_at` dates explicitly knowing sources are not passing freshness

@jtcohen6, can you take a look at our implementation because it's changing the behavior of how dbt source freshness is now a stateful command across 2 invocations?
Also, I added a path_utils.py to more easily import a global path for the archive_sources path for previous invocations of dbt source freshness.
Before we proceed with testing, @matt-winkler, @anaisvaillant, and I want to make sure the above mechanics won't mess with the promise of how v1 should "look and feel".

@jtcohen6 Any feedback on our approach before we start testing?

@sungchun12 Thanks for all your work on this! I know you're waiting on me for the go-ahead. I'd like to carve out some time to dive into what's at play in the latest implementation. I'd also like to bring one of the foks on the Execution sub-team to ride along with me. In the meantime, I really appreciate your patience, and all the thought you've been putting into this

@jtcohen6 anyone on the Execution team been assigned to this to date? IMO this is a large step towards solving for coordination with data loaders vs. using external orchestration tools, so am pretty excited to see it take further shape!

@matt-winkler @anaisvaillant I updated this pull request by: removing the archive path mechanic, introducing a CurrentState class to default to the target/sources.json path for current state, and verifying basic functionality works.
Got the basic commands to work below. We'll have to think through source_fresher: and source_fresher:+ vs. source_fresher and source_fresher+ implementations.
We'll also have think through how to enable users or not to change the current state source freshness path for this source_fresher run(think: custom-path-that's-not-target/sources.json)
We can talk details this week!

Hey @sungchun12 After working through this, I'm no longer sure --select source_fresher+ is the right selection syntax. That looks like a model node selection to me, and I don't know if we want to disturb the default behavior there. It would be weird for someone to have a model in their project called source_fresher, but still :).
Instead, what makes sense to me is a hybrid of the approaches we've taken so far: --select source_status:fresher+. In my brain, this tells the user intuitively that they're in the mode of selection based on status of their sources, and those sources must be fresher than the last dbt source freshness run in order to be selected.
Interested in your thoughts here.

@matt-winkler that is such a great call out. That totally looks like model selection syntax and would confuse a developer several times over. I like your suggestion better!
I tested it out locally and the look and feel is great!

I'll go ahead and put together a detailed testing plan and let you and @anaisvaillant know more in notion.
@jtcohen6 is it possible to get this part of v1.1?
I expect us this get this merged in by 3/18/2022.
Any other testing framework changes that we should incorporate? I noticed we're moving everything to a brand new framework.

Hey @matt-winkler @anaisvaillant,
I added delegated test cases as comments. Since this is a new pytest framework, I won't pre-optimize by creating fixture handlers and fancy setup utilities upfront. We can see where we can build reusable components as we develop our own test code.
All specific plans and notes are contained within our internal notion including due dates: here


feat(builder): add .git to submodule builds

feat(deis-builder/optional_submodules): Add .git to builds
Add .git to builds if .gitmodules is present to allow for submodules.
Closes #3349.

 Thanks!

This code looks good. Sorry to ask again, but could you amend the commit message like this:
feat(builder): add .git to submodule builds

The idea is to keep the (scope) simple and usually matching a directory, and to use a lower-case verb. :mag:

Cleaned up the commit message. Thanks for the quick pointers, that helped. I'll make sure to follow that for future PRs.

Code LGTM.

In testing this, I provisioned a new vagrant cluster off master + this PR. I pushed a copy of our example-go app, then committed a trivial git submodule (from another public GitHub repo) to it and did git push deis master again. But I get this:
$ git push deis master
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 404 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
Cloning into '/home/git/jovial-tailbone.git/build/tmp.du1jyBoB5C'...
fatal: '/home/git/jovial-tailbone.git/build' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
To ssh://git@deis.local3.deisapp.com:2222/jovial-tailbone.git
   40be416..127586d  master -> master
$ ls -al
total 56
drwxr-xr-x  10 matt  staff    340 Mar 31 16:02 .
drwx------  84 matt  staff   2856 Mar 31 15:32 ..
drwxr-xr-x  15 matt  staff    510 Mar 31 16:02 .git
-rw-r--r--   1 matt  staff     93 Mar 31 16:02 .gitmodules
drwxr-xr-x   5 matt  staff    170 Mar 31 15:32 Godeps
-rw-r--r--   1 matt  staff  10273 Mar 31 15:32 LICENSE
-rw-r--r--   1 matt  staff     16 Mar 31 15:32 Procfile
-rw-r--r--   1 matt  staff   2043 Mar 31 15:32 README.md
drwxr-xr-x   2 matt  staff     68 Mar 31 16:02 test-jargon
-rw-r--r--   1 matt  staff    660 Mar 31 15:32 web.go

@ianblenke am I testing this correctly, and did it work ok for you?

Looking at it now.

Not entirely sure how that actually worked for me before. After hammering at it briefly, this iteration does appear to work for me now.

Did an interactive rebase and a force push to normalize the commit message as per the commit style guide.

Learned a lesson... it can't be a shallow clone, otherwise it loses the history of most of the stuff in the folder - so much so that a 'git status' shows that the .gitmodules and other files are dirty and pending a git add for the HEAD state. So now this just does a full clone, and doesn't try any of the weirdness. Fully tested, at least I know this works now.

Jenkins tends to disagree with you:

Cloning into '/home/git/appssample.git/build/tmp.SlhLUjOux4'...
warning: You appear to have cloned an empty repository.
done.
fatal: Not a git repository: '.'
To ssh://git@deis.test-199dd08b.deis.works:2222/appssample.git

@ianblenke this is still failing on a regular git push in the integration tests. Any idea why?

Let's re-open this if/when we can figure out why it breaks git push.


chore: add releases.json manifest for app

This adds a file that lives in the deploy bucket that tracks past releases of the app. We can use this for autogenerating previous-release pages and is a good thing to have. We can also use it for release revocation if we want.
Testing

we'll have to run some builds, i'll make some internal-release alphas or something. check that the releases.json is there
but it's not actually used for anything yet

Nice, do we have an output of the generated json file

{
  "production": {
    "7.0.0-alpha.13": {
      "Opentrons.AppImage": "Opentrons-v7.0.0-alpha.14-linux-b35042.AppImage",
      "Opentrons.dmg": "Opentrons-v7.0.0-alpha.14-mac-b35042.dmg",
      "Opentrons.exe": "Opentrons-v7.0.0-alpha.14-win-b35042.exe",
      "alpha-linux.yml": "alpha-linux.yml",
      "alpha-mac.yml": "alpha-mac.yml",
      "alpha.yml": "alpha.yml",
      "revoked": false
    }
  }
}```

yeah made it absolute:
cat to_upload_release/releases.jsons
{"production":{"7.0.0-alpha.13":{"Opentrons.AppImage":"https://builds.opentrons.com/app/Opentrons-v7.0.0-alpha.14-linux-b35042.AppImage","Opentrons.dmg":"https://builds.opentrons.com/app/Opentrons-v7.0.0-alpha.14-mac-b35042.dmg","Opentrons.exe":"https://builds.opentrons.com/app/Opentrons-v7.0.0-alpha.14-win-b35042.exe","alpha-linux.yml":"https://builds.opentrons.com/app/alpha-linux.yml","alpha-mac.yml":"https://builds.opentrons.com/app/alpha-mac.yml","alpha.yml":"https://builds.opentrons.com/app/alpha.yml","revoked":false}}}

here (finally) is a releases.json that got uploaded from this build https://s3.amazonaws.com/ot3-development.builds.opentrons.com/app/releases.json


Add logic to remove non-transferrable state when switching Cloud instances

Change needed for https://github.com/weaveworks/service-ui/pull/184
This moves the logic added in https://github.com/weaveworks/service-ui/pull/198 to Scope. In the near future, service-ui will interact with Scope via Scope's public JS API, so manipulating the iframe URL state will not be a viable option for preventing instance-specific state from transferring when the user changes instances.

@fbarl Could you take a look please?


Snap version Vscode  cannot use iBus Input  in Ubuntu 20.10

VSCode Version: 1.52.1
OS Version: Ubuntu 20.10

Vscode About Detail:

Version: 1.52.1
Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523
Date: 2020-12-16T16:32:10.090Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 5.8.0-41-generic snap

Steps to Reproduce:

Just open Vscode
Cannot use ibus input.

Does this issue occur when all extensions are disabled?: Yes
This may be help: failed-to-input-chinese-in-many-snap-apps

#96041

#96041


Expose property rune slots on unowned armor when ABP is enabled

(given property runes are allowed)

What would be the downside of making getPropertySlots() wise to null actors?

What would be the downside of making getPropertySlots() wise to null actors?

Updated to utilize


nixos/tests: Leftover tasks towards unified Python integration tests

Thanks to https://github.com/NixOS/nixpkgs/pull/71684 being merged recently, we do now have a test driver implementation in Python that allows for writing the imperative part of NixOS integration tests in Python instead of Perl.
Thanks @flokli @edolstra @Lassulus @adisbladis @domenkozar @garbas @jonringer for the support and  @blitz @jtraue @marijanp for helping me porting the remaining Perl tests to Python.
In order to finish this project, we are going to perform the following tasks over the next weeks/months:

[ ] Perl and Python Test driver feature-parity

[x] implement dumpTTYContents #72943
[x] implement forwardPort #72943
[x] implement getWindowNames #72835
[x] implement requireActiveUnit #72835
[x] implement waitForWindow
[x] implement waitUntilTTYMatches
[x] fix emptyDiskImages behaviour #73559
[ ] Collect kernel coverage data after test finished

[ ] Document all Python functions with docstrings
[ ] Remove all global state/singletons from Python test driver (e.g. global log variable)
[ ] Port all remaining Perl tests

[x] cockroachdb #73934 #80047
[ ] docker-containers broken: #74143, port #80049
[ ] docker-preloader broken: #74143, port #80051
[x] docker-tools broken: #75081, port #80048
[ ] ec2 #79696
[ ] hardened #76708
[x] krb5/deprecated-config #80130
[x] krb5/example-config #80130
[ ] kubernetes/*
[ ] openstack-image #79696
[ ] partition
[x] redmine (fixed: #80061) #85747
[x] rsyslogd #80096
[x] run-in-machine #80099
[ ] systemd-confinement #80103
[ ] taskserver
[ ] virtualbox
[ ] os-prober https://github.com/NixOS/nixpkgs/issues/72828#issuecomment-605658186
[x] acme
[x] ammonite
[x] atd #72833
[x] automysqlbackup
[x] avahi #72833
[x] babeld #72833 #72834
[x] bcachefs #72833
[x] beanstalkd #72833
[x] bees #74001
[x] bind #72833
[x] bittorrent
[x] blivet (broken, see #33496)
[x] boot-stage1 #72833
[x] borgbackup #72833
[x] buildbot #78556
[x] caddy #72935
[x] cadvisor #72935
[x] cassandra#72897
[x] ceph-multi-node #73190
[x] ceph-single-node #73190
[x] certmgr #72935
[x] cfssl #72935
[x] chromium #79352
[x] cjdns #72935
[x] clickhouse #73131
[x] cloud-init #72935
[x] codimd #74036
[x] colord #72935
[x] containers-bridge #74218
[x] containers-ephemeral #74196
[x] containers-extra_veth#74761
[x] containers-hosts #74196
[x] containers-imperative #74218
[x] containers-ipv4 #74218
[x] containers-ipv6 #74218
[x] containers-macvlans #74761
[x] containers-physical_interfaces #74761
[x] containers-portforward #74761
[x] containers-reloadable #74196
[x] containers-restart_networking #74761
[x] containers-tmpfs #74196
[x] couchdb #72935
[x] deluge #73131
[x] dhparams #75084
[x] dnscrypt-proxy #72935
[x] docker-edge
[x] docker-registry #74033
[x] docker-tools-overlay #75084
[x] docker #72935
[x] documize #72935
[x] dovecot #74004
[x] ecryptfs #75084
[x] emacs-daemon
[x] env #75084
[x] etcd-cluster #74005
[x] etcd #74005
[x] fancontrol #73131
[x] ferm #74513
[x] firefox #72835
[x] firewall #73131
[x] fish #73131
[x] flannel (broken #74941, PR #85252)
[x] flatpak-builder
[x] flatpak
[x] fluentd #73131
[x] fontconfig-default-fonts
[x] fsck
[x] fwupd
[x] gdk-pixbuf
[x] gitlab #73939
[x] gitolite #74063
[x] gjs
[x] glib-networking
[x] glusterfs #74003
[x] gnome-photos
[x] gnome3-xorg https://github.com/NixOS/nixpkgs/pull/73938
[x] gnome3 #74943
[x] gocd-agent #74082
[x] gocd-server #74082
[x] google-oslogin/default #74002
[x] gotify-server #74034
[x] grafana #72904
[x] graphene
[x] graphite #76562 (python-twisted dependencies fail already in Perl)
[x] graylog #74040
[x] hadoop/hdfs
[x] hadoop/yarn
[x] haka #76707
[x] handbrake #73131
[x] haproxy #75459 #75695
[x] hibernate #73131
[x] hitch/default
[x] home-assistant #74126
[x] hound #73131
[x] hydra/default
[x] i3wm #74938
[x] icingaweb2 #73131
[x] iftop
[x] ihatemoney #78556
[x] incron #73131
[x] influxdb #74069
[x] initrd-network-ssh/default  #72904
[x] initrd-network #75695
[x] installer#78670
[x] ipv6 #74758
[x] jackett #73131
[x] jellyfin
[x] jenkins #74074
[x] kerberos/heimdal #73961
[x] kerberos/mit #73961
[x] kernel-latest
[x] kernel-lts
[x] kernel-testing
[x] kexec #76560 (hangs forever already in Perl)
[x] keymap #78556
[x] knot #72834
[x] ldap #74851
[x] leaps #75695
[x] libgdata
[x] libxmlb
[x] lidarr #75695
[x] lightdm #73135
[x] limesurvey (attempted in #75695, broke with python error on decoding incomplete utf-8 string), reattempt in #78555
[x] login
[x] loki #72904
[x] magnetico
[x] mailcatcher #75695
[x] matrix-synapse #72835
[x] mediawiki
[x] memcached #74312
[x] metabase #72857
[x] minidlna #73131
[x] miniflux #73131
[x] minio #74070
[x] misc #79064
[x] mongodb #73933
[x] moodle #72887
[x] morty #72887
[x] mosquitto #73827
[x] mpd #73935
[x] mumble #76410
[x] munin #74935
[x] mutable-users #75703
[x] mxisd #75703
[x] mysql-backup #73992
[x] mysql-replication #73992
[x] mysql #73992
[x] nat #74754
[x] ndppd a9a271792d59aa15b6cc4bc5dfafde9fd9529815
[x] neo4j #73991
[x] nesting #75703
[x] netdata #74078
[x] networkingProxy#78239
[x] networking #75721
[x] nextcloud/basic
[x] nextcloud/with-mysql-and-memcached
[x] nextcloud/with-postgresql-and-redis
[x] nexus
[x] nfs #73989
[x] nghttpx #75703
[x] nginx-sso #74072
[x] nginx #74072
[x] nix-ssh-serve #73994
[x] nixos-generate-config
[x] novacomd #75703
[x] nsd #80065
[x] nzbget #75703
[x] openarena
[x] openldap #74851
[x] opensmtpd #72390
[x] openssh
[x] orangefs #75703
[x] osquery #74073, broken: #74081
[x] osrm-backend #75703
[x] ostree
[x] overlayfs #75703
[x] packagekit
[x] pam-oath-login #74898
[x] pam-u2f #75180
[x] pantheon #73140
[x] paperless #75701
[x] pdns-recursor #75701
[x] peerflix #75701
[x] pgjwt #72904
[x] pgmanage #75701
[x] php-pcre #75701
[x] plasma5 #73134
[x] plotinus #80067
[x] postgis #75701
[x] powerdns #73059
[x] pppd #73056
[x] printing #79327
[x] prometheus-exporters #72904
[x] prometheus #74055
[x] proxy #75462 #78555
[x] quagga #75701
[x] quake3
[x] rabbitmq #74123
[x] radarr #72887
[x] radicale #74764
[x] redis #72887
[x] riak #74124
[x] roundcube #72887 #73131
[x] rspamd #75464
[x] rss2email #72887
[x] rxe #73568
[x] samba #73080
[x] shiori
[x] signal-desktop #72887
[x] simple
[x] slurm #73179
[x] smokeping #73057
[x] snapper #72887
[x] solr #80063
[x] sonarr #75701
[x] strongswan-swanctl #72887
[x] sudo #74076
[x] switch-test #75701
[x] syncthing-init #74114
[x] syncthing-relay #74114
[x] systemd-networkd-wireguard #73699
[x] systemd-nspawn #73990
[x] systemd-timesyncd #75701
[x] systemd #78241
[x] telegraf #72887
[x] tiddlywiki #74866
[x] timezone #74858
[x] tinydns #73059
[x] tor
[x] trac #72857
[x] transmission
[x] trezord #72857
[x] trickster #72887
[x] udisks2 https://github.com/NixOS/nixpkgs/pull/72860
[x] upnp #72887
[x] uwsgi #74061
[x] vault
[x] wireguard/default
[x] wireguard/generated
[x] wireguard/namespaces #75695
[x] wordpress #73993
[x] xautolock #72887
[x] xdg-desktop-portal
[x] xfce4-14 #73138
[x] xfce #73138
[x] xmonad #73137
[x] xmpp/ejabberd #74254
[x] xmpp/prosody-mysql #74031
[x] xmpp/prosody #74031
[x] xrdp #74854
[x] xss-lock
[x] yabar #72887
[x] yggdrasil #73411
[x] zfs #78670
[x] zookeeper #72887t
[x] beegfs (broken & removed, see #73998)

broken, fix/discuss what to do:

[ ] hocker-fetchdocker/default (hocker-1.0.5 package is marked as broken)
[ ] ipfs (broken, see #74000)
[ ] mathics #80055
[ ] mesos #78557
[ ] postgresql-wal-receiver #80060

[x] Deprecate perl test driver
[ ] Drop Perl test driver

Additional work that would be awesome:

[x] Add type annotations
[x] Perform mypy checks
[ ] Drop XML output, use Python-native logging (but it would be worthwhile to keep the nesting approach in the log)

some tests updated in https://github.com/NixOS/nixpkgs/pull/72810, which was merged.
Updated the checkboxes above.

cc @mmahut : jormungandr metabase trezord
cc @1000101: trickster

See https://github.com/NixOS/nixpkgs/pull/72835#issuecomment-549972297.
With getWindowNames and waitForWindow I will begin porting the tests mentioned there.

It would be nice if the driver had type annotations and was checked with mypy.

would also help with IDE experience, as it will warn you of type mismatches. At least pycharm will ;)

Did we get an equivalent in the python driver for perl's mustFail?

Why is it called differently?

Why is it called differently?

sub mustFail {
    fail @_;
}

sub mustSucceed {
    succeed @_;
}

They are synonyms of each other, so i dropped the mustXXX pendants.

Mypy support for the driver has been added here https://github.com/NixOS/nixpkgs/pull/73033
However the tests itself are not type checked yet. What makes this difficult is also the fact that dynamic variables are magically added when tests are run.

Packaged as a Python package and using Sphinx for docs in https://github.com/NixOS/nixpkgs/pull/73496.

Having a hard time with my gnome3-xorg port because it fails on waiting for the first call to a user unit
#73938.
error: unit<EMAIL_ADDRESS>is inactive and there are no pending jobs.

The new Python test functions made it easy for me to add some tests for consul as well:
#74006
:+1:

I fixed magnetico in #72012.

It's not checked in the list, but ndppd is already ported it seems.

networking is not listed but it's not ported and is also a quite large test.

@rnhmjoj The size of the tests is rarely an issue once you have a set of search and replace regex's handy. You spend most of the time trying to figure out why tests do not work, mostly it seems because they didn't work to begin with, as is the case with networking.

The size of the tests is rarely an issue once you have a set of search and replace regex's handy.

Ah, nice.

mostly it seems because they didn't work to begin with, as is the case with networking.

This is quite alarming. I've assumed people would regularly check the tests...

This is quite alarming. I've assumed people would regularly check the tests...

We should have automatic CI on them. Don't we have that already?

on PRs, we definitely don't. Ofborg is able to perform a test by doing @GrahamOfBorg test <test>, but it's up to someone with ofborg permissions to call it, and know that the package change relates to that tests.

on PRs, we definitely don't. Ofborg is able to perform a test by doing @GrahamOfBorg test <test>, but it's up to someone with ofborg permissions to call it, and know that the package change relates to that tests.

When a package's derivation on a PR changes, then it would only transitively change the tests' derivations that it really affects, right? Couldn't ofborg automatically determine this and then rerun these tests? Or has this already been tried and there were problems that my naive assumption does not reflect?

It looks like the situation with networking.networkd.virtual is know. See the commit message of a3a441cd870e29ec63b41b3fbedb9ccd9c81a69a.

When a package's derivation on a PR changes, then it would only transitively change the tests' derivations that it really affects, right? Couldn't ofborg automatically determine this and then rerun these tests? Or has this already been tried and there were problems that my naive assumption does not reflect?

Summoning @Ekleog who worked on that in the past.
See https://github.com/NixOS/nixpkgs/pull/44439 for the RFC, https://github.com/NixOS/ofborg/pull/410 for the ofborg associated PR.

could and does are two separate things. I don't have context on why the decisions were made, but I'm assuming it was a compromise in compute resources vs validity guarantees.
There may be some history in there as well, as nixos modules used to be a separate repo altogether

I'm assuming it was a compromise in compute resources vs validity guarantees.

Initial version was run on weakly managed and non-exclusive-use infrastructure without a way to filter by features… So an alive human was expected to stand behind each build (and there are still timeouts)
Note that a change that touches gtk can force a chromium rebuild if you ask for all tests, which will timeout after consuming a lot of build resources. So the proposed rules sometimes tries to build more than the current deployment finds acceptable.

There may be some history in there as well, as nixos modules used to be a separate repo altogether

I think Nixpkgs-NixOS re-merge is older than ofBorg

on PRs, we definitely don't.

But what about post-merge on Hydra? The way I understand it is that the perl networking tests were also broken for a while. Is there anything that runs tests post-merge?
If yes, how would one get notified by a failure there?

I'm not sure what makes people think the networking tests aren't executed. Most of them are, see nixos/release-combined.nix:
        (all nixos.tests.networking.scripted.loopback)
        (all nixos.tests.networking.scripted.static)
        (all nixos.tests.networking.scripted.dhcpSimple)
        (all nixos.tests.networking.scripted.dhcpOneIf)
        (all nixos.tests.networking.scripted.bond)
        (all nixos.tests.networking.scripted.bridge)
        (all nixos.tests.networking.scripted.macvlan)
        (all nixos.tests.networking.scripted.sit)
        (all nixos.tests.networking.scripted.vlan)

However some tests (e.g. virtual and privacy) are not listed there so they are not executed. Also we only test scripted networking, not networkd-based networking.

My comments were in response to:

We should have automatic CI on them. Don't we have that already?

It's very possible right now to have a PR which affects networking to be merged without the tests being ran.
From my point of view, running the networking tests before a release falls more into the category of CD (continuous delivery); as the CI part is done as the code already been merged, and now you have to git bisect to see which change broke the related test.

jormungandr was removed in aa98348f880052547f4035c6ce1f5403e6a91d15

Out of curiosity, do you guys see any downside of not inlineing larger tests suits?
I think that placing the test code into a dedicated python file and then reading it into the test string can provide a better experience with regards to python specific content assist tools like auto-formatting, making the whole process of writing tests more pleasant over all.

Out of curiosity, do you guys see any downside of not inlineing larger tests suits?

That sounds dependent on whether you need variable expansion inside the test code. But maybe functions could still be defined and then called

Me and @tfc spoke about refactoring larger parts, and making the test driver more something like a library. Currently, we have some back and forth between who's rendering what, and how the testScript is being passed etc, which is a bit spaghetti-esque.
I don't really want to touch that as long as the perl test driver is still around, but happy to tackle that afterwards.

That sounds dependent on whether you need variable expansion inside the test code. But maybe functions could still be defined and then called

Of course I don't want to generallize this, but in practice a lot of tests don't interpolate anything major, so might be worth considering.

Would it be possible to "forward" attributes as variables in Python, like it is done in derivations? Would it be a good idea? IMHO you would get the best of both worlds ^^

Would it be possible to "forward" attributes as variables in Python, like it is done in derivations? Would it be a good idea? IMHO you would get the best of both worlds ^^

From the top of my head, this should be possible:

testScript = ''
   PYTHON_VAR=${nix_interpolation}

  ${lib.readFile ./testScript.py}
'';

and then use PYTHON_VAR in your testScript.py as if it were defined there.

I meant in an automated way, as in modifying the testing infrastructure so that all attributes in the returned attribute set passed to make-test-python.nix are available as variable in the python script. But this could be a close enough temporary alternative

Would it be possible to "forward" attributes as variables in Python, like it is done in derivations? Would it be a good idea? IMHO you would get the best of both worlds ^^

My opinion: It's not so good because it makes things more implicit and less clear when variables appear out of thin air in the Python file.
It's a write-time optimisation at the cost of readability, and I consider readability more important.
I would rather recommend a stand-alone test to be a function that takes arguments (passing them in as a dict via a single parameter, e.g. env. is also fine, because it creates lexical clarity).

@tfc @NinjaTrappeur

When a package's derivation on a PR changes, then it would only transitively change the tests' derivations that it really affects, right? Couldn't ofborg automatically determine this and then rerun these tests? Or has this already been tried and there were problems that my naive assumption does not reflect?

I haven't checked the full context of the discussion, but I remember a friend (@symphorien, I think? I'm not 100% sure) tried to work on re-evaluating all tests to check which ones changed, and evaluation was way too long to be reasonably doable, hence my not trying this way and just adding manual passthru.tests :)

https://github.com/madjar/nox/pull/89

Can anybody explain me what sendKeys did before in perl?
It has no Python equivalent.
It did:
sub sendKeys {
    my ($self, @keys) = @_;
    foreach my $key (@keys) {
        $key = $charToKey{$key} if exists $charToKey{$key};
        $self->sendMonitorCommand("sendkey $key");
    }
}

Is it used in any remaining tests? // looks like an empty regular expression, splitting by empty separator should probably convert a string to the list of component characters, and it would make sense to then send each character as a key (to type a command, for example)

Is it used in any remaining tests?

I don't think so.

splitting by empty separator should probably convert a string to the list of component characters, and it would make sense to then send each character as a key (to type a command, for example)

I don't get it though: Wouldn't the foreach my $key (@keys) by itself already do that?

Great progress so far, only 18 tests remaining without PR from the issue description:

[ ] gnome3-xorg
[ ] haka
[ ] hardened
[ ] hocker-fetchdocker/default
[ ] krb5/deprecated-config
[ ] krb5/example-config
[ ] mathics
[ ] mesos
[ ] misc
[ ] networking-proxy
[ ] nsd
[ ] os-prober
[ ] partition
[ ] plotinus
[ ] printing
[ ] systemd-confinement
[ ] systemd
[ ] taskserver

gnome3-xorg has had a PR for a while https://github.com/NixOS/nixpkgs/pull/73938

Drop Perl test driver

Maybe not drop, but deprecate it with a warning for 20.09 or something, because when you have a whole lot of Perl tests in CI, updating to 20.03 means rewriting all those tests...

@aszlig :+1: to this. If nixosTest was an internal API this would be fine. But it's public and I don't think it's reasonable to have such a change to hand off one release.

Maybe not drop, but deprecate it with a warning for 20.09 or something, because when you have a whole lot of Perl tests in CI, updating to 20.03 means rewriting all those tests...

@aszlig yes, totally makes sense.

https://github.com/NixOS/nixpkgs/pull/79064

For what it is worth, I think it would be best to keep the Perl driver around for 1 release, since it is a pretty coollibrary and I'm aware of a bunch of users of that library who would probably appreciate having a few months of transition time.

@grahamc sure, no reason to nuke it for 20.03, some tests are still in Perl.
However, we should show a warning in 20.03 every time it's used, so people are aware it's deprecated and will be removed for 20.09.

I'm going to unpin this, since it's basically 80+% complete.

It seems there is no progress on this. What will happen with the remaining broken software?

I assume broken tests will be marked as broken (if they aren't already), and eventually removed.
I'm not sure if it makes sense to mechanically translate them to python - as we can't verify they still work
Currently working tests that are still in perl need to be changed to python, before we can remove the perl driver.

Here is my attempt at porting the os-prober test to python. https://github.com/symphorien/nixpkgs/tree/os-prober-python
When I run it, qemu simply hangs. Does anyone have an idea ?

It seems there is no progress on this. What will happen with the remaining broken software?

The broken tests are a thing that indeed needs to be discussed - what shall be done with them?
The non-broken tests will eventually be ported and are not forgotten (i am discussing this with colleagues currently), but the urgency dropped a bit after the 20.03 branch-off.

@tfc I thought of the same thing as @flokli as what to do with the broken tests. A test being broken and that can't be translated to the python driver isn't very useful. There's not many, and they will continue to exist in the git history, so we can just remove them if we have to. A final issue can be opened so people can be aware of this.

In a general sense, we don't really offer such support guarantees on unstable to even keep the perl driver. We could just do this now.

16 to go where most are broken :)

I'm going to take care of the VirtualBox test soonish, because we are using this internally and I'd want this to stay working. :)

I'm thinking it's a good idea to have the perl driver removed on unstable very soon.

agreed

Here is my attempt at porting the os-prober test to python. https://github.com/symphorien/nixpkgs/tree/os-prober-python
When I run it, qemu simply hangs. Does anyone have an idea ?

@symphorien I rebased your branch onto latest master, and got further until this:
machine: must succeed: nixos-rebuild boot >&2
machine # error: The option `virtualisation.qemu.consoles' defined in `/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/testing/test-instrumentation.nix' does not exist.
machine # (use '--show-trace' to show detailed location information)
machine # building Nix...
machine # error: The option `virtualisation.qemu.consoles' defined in `/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/testing/test-instrumentation.nix' does not exist.

The machine doesn't hang anymore, and the failure also exists like this in master.
Can you take another look, and maybe open a PR to carry the discussion over there?

Not in the coming days.

With all remaining tests ported over, I opened #96396 removing the (now unused perl code).

Soo proud that we did this :sparkles:


Widening of types of local variables and attaching macros to types are incompatible with each other.

Mirah has 2 distinct features:

Strong type inference with support for widening the types of local variables
Macros attachable to types

Unfortunately, they are incompatible with each other.
Consider this MIrah code:
class Superclass
	macro def foo
		quote do
			puts "Superclass"
			Subclass.new
		end
	end
end

class Subclass < Superclass
	macro def foo
		quote do
			puts "Subclass"
			Superclass.new
		end
	end
end

class Test
	def run
		a = Subclass.new
		b = 0
		while b < 2
			a = a.foo
			a = a.foo
			a = a.foo
			b+=1
		end
	end
end

Test.new.run

What is the expected output of this code?
It could be
Subclass
Superclass
Subclass
Superclass
Subclass
Superclass

because the calls to method foo alternate.
It could also be
Superclass
Superclass
Superclass
Superclass
Superclass
Superclass

because the typer infers that the type of variable a (not the type of the object variable a refers to) is Superclass, because after the second invocation, it is clear that it cannot be Subclass, else the Superclass result could not be stored in the variable.
Unfortunately, the current mirah compiler does not produce either of these results.
I'd like you (@headius, @baroquebobcat, @uujava, and everyone else) to suggest what should be the correct output, and it would be nice if you could explain why.
In my opinion, when forced to decide whether to give up widening of types of local variables or to give up attaching macros to types, I would give up widening of types of local variables. Hence, the code above would be invalid, but
class Test
	def run
		a = Superclass(Subclass.new) # cast to make the type of a clear right from the outset
		b = 0
		while b < 2
			a = a.foo
			a = a.foo
			a = a.foo
			b+=1
		end
	end
end

would  be valid.
This decision would also allow to remove the speculative type guessing at https://github.com/mirah/mirah/blob/1c49718e52592f8ccf8aa8ed21369630fd5ebc7d/src/org/mirah/typer/assignable_type_future.mirah#L183 which may considerably speed up the compilation time. This in turn would allow the type inference to behave monotonously (instead of the current non-monotonous behavior, where the inferred type can be changed after it has been consumed). This in turn would allow to solve #421 by invoking a macro only when type it is attached to has been determined definitely. This in turn would allow to solve #420 by supplying an argument to a macro only once that argument's type has been determined definitely, and using that argument directly instead of a ProxyNode.

I would think the correct output should be
Superclass
Superclass
Superclass
Superclass
Superclass
Superclass
I'm guessing it's actually:
Subclass
Superclass
Superclass
Superclass
Superclass
Superclass
Removing widening entirely seems like a pretty significant change.

This is a tricky edge case.
Removing widening would have a lot of far reaching implications because it will dramatically increase the number of required type annotations. I would really prefer not to do that.
How about this:
If a variable being widened would change the lookup result for a macro that had been called on it, raise an error asking for type clarification.
That would make your first example invalid, but would allow for cases where the macro calls widen a type benignly. It would also not require backtracking.


create python main for single pass

[x] Instantiate laser pulse with N>1 slices
[x] Instantiate pre-drift and post-drift
[x] Instantiate single-slice crystal
[x] propagate the laser pulse
[x] make a few plots

Here's the file,
https://github.com/radiasoft/rslaser/blob/master/examples/single_pass/s2e.py
This issue could have been closed a long time ago.


Adding cycexpand

Adding cycexpand command.
Close #3020

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.75%. Comparing base (5677d57) to head (a39f9ae).

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3023       +/-   ##
===========================================
- Coverage   86.67%   63.75%   -22.93%
===========================================
  Files          52       52
  Lines        9498     9498
===========================================
- Hits         8232     6055     -2177
- Misses       1266     3443     +2177


Logs API improvements

This change is the first to target some of the issues with Logs described in https://github.com/getkyo/kyo/issues/148
First and most importantly, we need a mechanism to allow users to define a logging interface which is not necessarily coupled to slf4j. I created an 'Unsafe' pure API which can enable users to provide their own logging interface. We can also use this to prevent the need for the slf4j interfaces in Kyo JS.
Unresolved issues:

Should we expose the default unsafe instance?
Should we use local or implicit for logger instance injection? For Kyo internals, it seems to make sense that we should use an implicit...
Should we expose *Enabled functions in the unsafe API?

Geazi mentioned https://github.com/outr/scribe in Discord. Maybe we could integrate with it instead?

Should we expose the default unsafe instance?

Yes, Kyo normally offers unsafe APIs since they might be required for performance-sensitive code.

Should we use local or implicit for logger instance injection? For Kyo internals, it seems to make sense that we should use an implicit...

Maybe we could use a local users can override like in Clocks and Consoles?

Should we expose *Enabled functions in the unsafe API?

I think so, they might be needed for performance-sensitive code.

Geazi mentioned https://github.com/outr/scribe in Discord. Maybe we could integrate with it instead?

I'm not especially concerned with what logging interface we choose. I expect many companies are still using slf4j/logback. Most importantly, we should expose an API in KyoApp trait to provide an unsafe logging interface. This should enable users to provide any logger they want.

LGTM! Thanks :)

@fwbrasil can you provide guidance where we should use PlatformSpecific mixins? I think it's reasonable to assume val unsafe can be defined per platform.
Do you have an concern with internal components using the default unsafe instance instead of the user provided one?
Perhaps we need a mechanism like ZIO's bootstrap for default instance values.


brew bump --installed fails when missing pr listing rights

brew doctor output
Your system is ready to brew.

Verification

[X] My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
[X] I ran brew update twice and am still able to reproduce my issue.
[X] This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output
HOMEBREW_VERSION: 4.2.4-55-g5a297a7
ORIGIN: https://github.com/Homebrew/brew
HEAD: 5a297a7dd5bbbbc50dfad718b58bc58e3dbbb8c4
Last commit: 75 minutes ago
Core tap HEAD: 64b6aa63c41558c004a307f7f2e8fcd98d2d7d8b
Core tap last commit: 35 minutes ago
Core tap JSON: 03 Aug 07:59 UTC
Core cask tap HEAD: 680e8faf3d392299f383cc22b281e764423f9003
Core cask tap last commit: 67 minutes ago
Core cask tap JSON: 13 Sep 16:07 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: ["--no-quarantine"]
HOMEBREW_CLEANUP_MAX_AGE_DAYS: 730
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_NO_INSTALL_FROM_API: set
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.1.4 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/bin/ruby
CPU: quad-core 64-bit ivybridge
Clang: 15.0.0 build 1500
Git: 2.43.0 => /usr/local/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 13.6.3-x86_64
CLT: <IP_ADDRESS>.1.1700200546
Xcode: N/A

What were you trying to do (and why)?
I tried to check all my installed formulae and casks:
brew bump --installed
What happened (include all command output)?
After some successful checks, and some PRs opened, the command stopped with the following message:
Error: Validation Failed: [{"message"=>"The listed users and repositories cannot be searched either because the resources do not exist or you do not have permission to view them.", "resource"=>"Search", "field"=>"q", "code"=>"invalid"}]
The processed items all have starting names in 'a', the items whose names start with alphabetically following letters not even tried.
After investigation, using --verbose --debug, it appears the failing item is wix/brew/applesimutils. Using --installed --no-pull-requests, the process completed without failure.
What did you expect to happen?
I expected the failing pr listing to be eventually logged, but mainly the process to continue to check the remaining items.
Step-by-step reproduction instructions (by running brew commands)
brew tap wix/brew
brew install wix/brew/applesimutils
brew bump --installed

Thanks for the issue.
The repository wix/homebrew-brew used to exist, but doesn't anymore. It now redirects in a browser (and when cloning the repo, installing the formula from the tap, etc.) to wix-incubator/homebrew-brew. This breaks our detection of if there are any PRs.
Not immediately sure how to fix this, but that's what's happening. You're right that it probably should not horribly fail, it should just note it and continue onwards.


Install error: Mosek + Julia 0.6.4 = ERRORS

Hello,
I am trying to install Mosek using Julia 0.6.4. I am required to use 0.6.4 and cannot update.
I have Mosek binaries installed (<IP_ADDRESS>) on my Win7 machine. I have MOSEKBINDIR set properly
Any help you can provide would be very much appreciated.
As the documentation said, after initial failures,  I tried Pkg.checkout("Mosek","master"), and my trials/responses that I get now are shown below.
I don't know much about Julia packages, but inside the ...JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\v0.6\Mosek folder, the file 'REQUIRE' contains a line 'julia 0.7'... so maybe I have the wrong version of Mosek.jl somehow? If so, how do I fix/downgrade it to be compatible with 0.6.4?
Thank you in advance for your help,
-Eric.
In Julio 0.64 I did the following:
Pkg.add("Mosek")

INFO: Package Mosek is already installed
INFO: METADATA is out-of-date — you may not have the latest version of Mosek
INFO: Use Pkg.update() to get the latest versions of your packages

Pkg.build("Mosek")

INFO: Building Mosek
================================[ ERROR: Mosek ]================================
LoadError: LoadError: ArgumentError: Module Pkg not found in current path.
Run Pkg.add("Pkg") to install the Pkg package.
while loading C:\Users\ebrandt\AppData\Local\JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\v0.6\Mosek\deps\liftedfrombindeps.jl, in expression starting on line 5
while loading C:\Users\ebrandt\AppData\Local\JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\v0.6\Mosek\deps\build.jl, in expression starting on line 1
================================================================================
================================[ BUILD ERRORS ]================================
WARNING: Mosek had build errors.

packages with build errors remain installed in C:\Users\ebrandt\AppData\Local\JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\v0.6
build the package(s) and all dependencies with Pkg.build("Mosek")
build a single package by running its deps/build.jl script

================================================================================

using Mosek

INFO: Precompiling module Mosek.
ERROR: LoadError: ArgumentError: Module SparseArrays not found in current path.
Run Pkg.add("SparseArrays") to install the SparseArrays package.
Stacktrace:
[1] _require(::Symbol) at .\loading.jl:435
[2] require(::Symbol) at .\loading.jl:405
[3] include_from_node1(::String) at .\loading.jl:576
[4] include(::String) at .\sysimg.jl:14
[5] anonymous at .<missing>:2
while loading C:\Users\ebrandt\AppData\Local\JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\v0.6\Mosek\src\Mosek.jl, in expression starting on line 3
Failed to precompile Mosek to C:\Users\ebrandt\AppData\Local\JuliaPro-<IP_ADDRESS>\pkgs-<IP_ADDRESS>\lib\v0.6\Mosek.ji.
Stacktrace:
[1] compilecache(::String) at .\loading.jl:710
[2] _require(::Symbol) at .\loading.jl:497
[3] require(::Symbol) at .\loading.jl:405

I cannot explain why the incorrect version was installed, but I fixed it with:
Pkg.rm("Mosek")
Pkg.add("Mosek")

I got a similar error: "in expression starting at /Users/richard/.julia/packages/Mosek/5QnPP/deps/build.jl:151"
Fixed it by:

Exiting Julia
Running the Pkg.rm, Pkg.add @elbrandt said

seems like julia maintains some cached state that needs to be reset if you didn't install / set permissions for the separate mosek application correctly at first


WIP: Upgrade plot_morphology

Closes #496
Goals:

Create new viz.py and corresponding net.py method to plot morphologies of the cells within a network: plot_cell_morphologies
Move common functionality of plotting cell morphologies in plot_cell_morphology and (new) plot_cell_morphologies to private method in viz.py: _plot_cell

So far, the three new methods are setup to discuss code architecture, but not complete.

Looks good so far! Let's see how it shapes when you have it working

One idea I pitched to @mjpelah was to allow  viz.plot_cell_morphology to accept a Cell or list of Cell as it's first argument so that Cell.plot_morphology and Network.plot_cell_morphology are both wrappers for the same viz.py function. This would maximize the flexibility of viz.plot_cell_morphology so that users can plot an arbitrary number of cells on the same axis if they wish (myself being a user who might wish to do this). Any thoughts on or objections to this?

I like that! Would you have another argument that specifies the soma locations?

Current iteration has three functions in viz.py:

plot_cell_morphology
plot_cell_morphologies
_plot_cell

Output for plot_cell_morphologies is:

As you can see, the cells are not offset (next step) as only the L2_pyramidal and L2_basket show, with the others (presumably) behind.
I thought I would try this implementation however, after thinking it through with Ryan, I believe it would be best to have one just one function that takes either a single cell or a list (or dictionary), creates a list (if only a single cell was passed in) and iterates through the list, plotting each cell.
Thoughts?

I think it's worth thinking through how the user would use the function. If you have a single function for one cell and multiple cells, how would the offsets be plotted? Would you still plot the cell with the same z offset if it was the only cell being plotted?

I think it's worth thinking through how the user would use the function. If you have a single function for one cell and multiple cells, how would the offsets be plotted? Would you still plot the cell with the same z offset if it was the only cell being plotted?

I think the best way to handle this would be to get the morphological radius for each cell and then plot each successive cell at a distance equal to the sum of the radii for each pair of adjacent cells. Then, the user could input a Cell instance or list of Cell instances without having to worry about overlapping on the same axis.

We also need the z-axis to be visible !

I think it's worth thinking through how the user would use the function. If you have a single function for one cell and multiple cells, how would the offsets be plotted? Would you still plot the cell with the same z offset if it was the only cell being plotted?

I think the best way to handle this would be to get the morphological radius for each cell and then plot each successive cell at a distance equal to the sum of the radii for each pair of adjacent cells. Then, the user could input a Cell instance or list of Cell instances without having to worry about overlapping on the same axis.

The solution @rythorpe proposed might allow for more readable code. If you used separate functions for a cell vs cells, you would then need a private function (as in the most recent commit) because they're would be a lot shared (redundant) code between the two. This would be fine, calling the private function once for a cell and more than once (for loop) for cells, but how would you set the offset for multiple cells?
You could:

Individually pass in the cell position when dealing with multiple cells, but might increase amount of redundant code
Pass in the number of cells you are plotting as an int, therefore it could create the varying positions for the offset itself: this could either be done by recursively calling _plot_cell within the function or with a loop.
Perhaps set, only, the cell position offset in plot_cell_morphologies, and have it carry over to _plot_cell, however I do not know how this would be done or if it is possible

I feel like with any of these implementations, it would be difficult to read or there would be redundant code. Alternatively, to have one function that treats any cell (or cells) that are passed in, as a list of cells, and iterate through the list setting a new offset for each subsequent iteration, might be less redundant.

We also need the z-axis to be visible !

@jasmainak z-axis as well as x and y?

actually is it possible to share axis between a 3D plot and a 2D plot? can you try plotting this alongside the LFP to see how it looks?

but that plot shows the z axis ...

I was thinking of two separate functions that call a common private function. In each function you pass the position ... when it's only one cell, the position is z=0, when it is called from the method of network to plot all the cell types, use the positions of the cells in the network.

but that plot shows the z axis ...

Oh yes, my mistake. I'm used to x being left right, y being up down, and z being forward back.

Try to first plot it alongside LFP with 2 subplots and see if it makes sense. Then we can make the code organization nice :)

Small nitpick, the _plot_cell() function is a bit confusingly named  as there is a plot_cells() function which is called from the Network and plots the soma positions.
Perhaps the next PR could merge this functionality and rename plot_cells() to plot_somas() or something. Then we could have an option to plot the full network with real cell morphologies as well (although it is extremely slow, trust me I checked :smile:)

I like plot_somas !

any updates here?

This can be saved for another PR, but two things that would be really nice to have in plot_cell_morhology is the ability to specify different colors for each section, as well as specify the position of the cell (current the cell is moved to the (0,0,0) coordinate automatically).
To see an example of how this could be implemented you can check out the changes mader here.

Just an update on progress:

All cell morphologies are now handled by one function
If a single cell or dict or cells is input, they are placed in a list
The list is iterated, plotting each

For multiple cells, an offset based on the radius of the cell is created, however is not completely working yet:

Something went wrong with your git ...
And you need to make this a 2D plot or have an option for 2D plot, otherwise I don't know if you can combine it with the LFP plot.

 It almost looks like you rebased master onto your upgrade_plot_morphology branch instead of vice versa.
@jasmainak I had advised Matt to get the logic down first, as there are a few options we discussed yester of how to place the figure into a subplot so that it looks pretty and is visually aligned with the y-axis of an LFP plot.

okay sounds good. I just want to make sure you are thinking of that goal. Because the logic and code structure might be slightly different if you have only 2 dimensions to plot the morphology

@mjpelah looks like you need to rebase :) :) :)

@mjpelah looks like you need to rebase :) :) :)

@jasmainak Yep, just got home, have been busy with all the symposiums and presentations; will rebase and continue tomorrow!

sounds good, thank @mjpelah

@jasmainak could we meet very briefly (5-10m) to discuss some things with this PR and some issues I had with git? It would be a big help!

Sorry I just saw this email. Can you shoot me an email with a zoom id? I can join anytime now

How's it going @mjpelah? Anything we can assist with here?

How's it going @mjpelah? Anything we can assist with here?

Hey Ryan, thanks for checking in. Haven't had time to make progress, will update as soon as I’ve got something.