FR: Server setup hooks

First off, I'd like to express my appreciation for the work already done. I am extremely glad this extension exists and providers the first-party one some much needed community competition so thank you for your work! Even if this feature request is rejected this extension is already very useful and I know how to work around the limitations I am facing in my personal use-case.
As to the feature request, I'd like to be able to hook into the install script to customize the installation process on SSH connections where the default approach does not work for whatever reason.
Personally, I would like to run a VSCode server in a container managed by Nix and the node binary that comes bundled with the server does not work in Nix systems.

The NodeJS by default supplied by VS Code cannot be used within NixOS due to missing hardcoded paths.
https://github.com/nix-community/nixos-vscode-server

There are multiple ways to solve this, in fact, the quote above comes from a project that solves this issue in NixOS, but I think the cleanest solution would be one powered by this extension itself and on top of solving the problem I'm having, it would open up possibilities to support more types of systems and prevent issues such as #116.
Personally I am looking only for one hook that is called after the server is installed and before it is attempted to start, perhaps patchServer would be a nice name for it. I would use it to rm the bundled node and ln one from the Nix store, but I imagine being able to overwrite how the server is downloaded (installServer) and started (startServer), or even a hook for after shutting it down (afterStopServer), would be very useful to some people. #116, for example, could be solved by replacing installServer hook to not use the missing wget options. On top of that, this would probably also make it possible to remove some options, like serverDownloadUrlTemplate to reduce technical debt.
I realize that there is currently no way to configure anything per-connection other than the SSH config and configuring this in the extension would probably be hard to develop and maintain so I would suggest to read this configuration from the remote shell, maybe just source .vscode-server/open-remote-ssh/hooks.sh to give the remote a chance to control how the server is managed on it?
Please let me know if you are interested in adding this and need any assistance or additional input. I've never built VSCode extensions before but can contribute by testing and trying various hook configs or, if needed, I am sure I can figure out how to develop a VSCode extension too.

I just realized that this feature could also solve #35. The users could modify the installServer or another hook before that, maybe beforeInstallServer or confiureInstall to either install to a shared directory or change SERVER_DATA_DIR and then modify startServer accordingly if needed

I agree that if the server variables are set before and used afterwards, this could solve #35 as a workaround (but an extension setting to have a configurable target directory that will also be used during "preparation" will work better for most users).
I definitely see this FR as a useful option also for other workarounds, and especially for replacing not-working node-versions, like here (docker) or #127 (which includes the workaround - a manual ssh after installation, then reconnect via the extension).
Would you place this server setup hook in the ssh config "somehow" or intend to have it as an extension setting (in which case it has to go into an array that includes the target name, because the hooks will be different per target)?

I have already made a fork where something similar is implemented that I am using in a highly experimental setup. My solution allows the remote to completely take over the server setup process.
Currently my fork breaks compatibility with Microsoft VS Code (so it is Codium-only) and unimlpements custom links.
I am planning to improve my fork and offer the change to the maintainer


"Run a Model" section clarifications

First I think it's important to let the user know where he'll find the scripts used to run models. This depends on whether or not they've setup a virtual environment. I have and mine are in ./venv/bin.
Second, it's critical (IMO) that we explain that the directory from which these credmark-dev (and possibly other commands) is run matters. For example, if I run credmark-dev list it from the bin directory, this is the output:
Loaded models:

 - series.block-start-end-interval: ['100.0']
 - series.block-window-interval: ['100.0']
 - series.time-start-end-interval: ['100.0']
 - series.time-window-interval: ['100.0']

0 errors, 0 warnings

If, on the other hand, I run it from the root directory, i.e., ./venv/bin/credmark-dev list, this is the output:
Loaded models:

 - aave.lending-pool-assets: ['1.0']
 - aave.overall-liabilities-portfolio: ['1.0']
 - aave.token-asset: ['1.0']
 - aave.token-asset-historical: ['1.0']
 - aave.token-liability: ['1.0']
 - account.portfolio: ['1.0']
 - cmk.circulating-supply: ['1.0']
 - cmk.total-supply: ['1.0']
 - compound-token-asset: ['1.0']
 - compound-token-liability: ['1.0']
 - compound.test: ['1.0']
 - contrib.neilz: ['1.0']
 - contrib.neilz-redacted-cashflow: ['1.0']
 - curve-fi-all-gauge-addresses: ['1.0']
 - curve-fi-all-pool-info: ['1.0']
 - curve-fi-all-yield: ['1.0']
 - curve-fi-avg-gauge-yield: ['1.0']
 - curve-fi-get-gauge-stake-and-claimable-rewards: ['1.0']
 - curve-fi-historical-lp-dist: ['1.0']
 - curve-fi-pool-historical-reserve: ['1.0']
 - curve-fi-pool-info: ['1.0']
 - curve-fi-pools: ['1.0']
 - example.address-transforms: ['1.0']
 - example.blocktime: ['1.0']
 - example.echo: ['1.0']
 - example.historical: ['1.0']
 - example.historical-block: ['1.0']
 - example.historical-block-snap: ['1.0']
 - example.historical-snap: ['1.0']
 - example.iteration: ['1.0']
 - example.ledger-blocks: ['1.0']
 - example.ledger-contracts: ['1.0']
 - example.ledger-logs: ['1.0']
 - example.ledger-receipts: ['1.0']
 - example.ledger-token-transfers: ['1.0']
 - example.ledger-tokens: ['1.0']
 - example.ledger-traces: ['1.0']
 - example.ledger-transactions: ['1.0']
 - example.libraries: ['1.0']
 - example.load-contract-by-address: ['1.0']
 - example.load-contract-by-name: ['1.0']
 - example.run-test: ['1.0']
 - example.token-loading: ['1.0']
 - example.type-test-1: ['1.0']
 - example.type-test-2: ['1.0']
 - finance.lcr: ['1.0']
 - price: ['1.0']
 - series.block-start-end-interval: ['100.0']
 - series.block-window-interval: ['100.0']
 - series.time-start-end-interval: ['100.0']
 - series.time-window-interval: ['100.0']
 - sushiswap.all-pools: ['1.0']
 - sushiswap.get-average-price: ['1.0']
 - sushiswap.get-pool: ['1.0']
 - sushiswap.get-pool-info: ['1.0']
 - sushiswap.get-pools: ['1.0']
 - token.categorized-supply: ['1.0']
 - token.holders: ['1.0']
 - token.info: ['1.0']
 - token.overall-volume: ['1.0']
 - token.swap-pool-volume: ['1.0']
 - token.swap-pools: ['1.0']
 - uniswap-v2.get-average-price: ['1.0']
 - uniswap-v2.get-pools: ['1.0']
 - uniswap-v3.get-average-price: ['1.0']
 - uniswap-v3.get-historical-price: ['1.0']
 - uniswap-v3.get-pool-info: ['1.0']
 - uniswap-v3.get-pools: ['1.0']
 - uniswap.exchange: ['1.0']
 - uniswap.quoter-price-usd: ['1.0']
 - uniswap.router-price-usd: ['1.0']
 - uniswap.tokens: ['1.0']
 - xcmk.cmk-staked: ['1.0']
 - xcmk.deployment-time: ['1.0']
 - xcmk.total-supply: ['1.0']

1 errors, 0 warnings

Also, I don't think it's great that the example we provide (with no explanation of arguments 😬) requires a web3 provider. Maybe we could suggest one of each.

This seems more of a bug. if you run credmark-dev list command in any of subfolder (or any other folder other than root folder) it shows below output:

series.block-start-end-interval: ['0.0']
series.block-window-interval: ['0.0']
series.time-start-end-interval: ['0.0']
series.time-window-interval: ['0.0']

FYIP
@iwyrkore @newtectonics

This is not a bug.  The folder where you run the command does not matter. The default model folder is "models" and it looks for that in the current folder. Maybe that can be more clear in the docs. Also you can use the command-line flag to change that to another folder/path.
The path of credmark-dev is taken care of in the python install and you don't need to know the path to use it. Python will have it in your search path. This is standard python stuff -- we don't choose where to install it.

Updated docs.


Update console apps and web site projects to .NET Core

Currently, all console service host apps and web site projects in excercises are using full .NET Framework. It should not be very hard to switch them to .NET Core.

Hey @alexeyzimarev thanks for raising this. It’s in the plans, we’re simply waiting for NServiceBus on Net Core to be at least RC to migrate everything to .NET Core

May be changing project file and adding <Frameworks>net461</Frameworks> to use full .NET Framework, would do for a moment?

Is this also blocked while we want to keep supporting VS2015?

Yeah, I am closing this one because of VS 2015.

No problem @alexeyzimarev. At some point we will do this though, as soon as a) we decide to make VS 2017 the minimum requirement and b) NServiceBus 7 is in RC/RTM.


Explore analytics (Mixpanel.set) not working

So I am trying to use the Explore section on Mixpanel but I need to identify and set user properties. The problem is my users are anonymouse so I just want access to the explore tab via users' disctinct ID.
I am calling identify and set but it is not working:
   Mixpanel.identify();     Mixpanel.set({"emailtest":"emailtest"});
I have also tried to call set without any parameters but it still doesn't work. I still get this message on Mixpanel: "You haven't sent us any user profiles yet. Don't worry: it's easy to get set up."
Everything else works fine. Any help will be appreciated :D

Try something like this:

Mixpanel.getDistinctId((id) => {
  Mixpanel.identify(id);
 Mixpanel.set({"emailtest":"emailtest"});
});


Typo in Paper

In Figure 4, there is a typo in the word "green". Great work team!

Thanks so much for your careful reading! We appreciated your efforts and will fix this in the next version arXiv.


Completed/improved Dutch translations

Added documentation on how to build/run the application on Linux (both Debian and OpenSUSE), as some packages need to be installed.
Improved some existing translations so they sound more natural
Added missing translations
Changed title-settings-style to the English word "stylesheets". Reason for this is that "Aangepaste stijlen", as well as not really sounding natural, caused issues in the menu due to its length. Stylesheets, while being an English term, will be understood by any Dutch speaker who is looking to load custom CSS files.

Seems like something is wrong with the MacOS CI :) https://circleci.com/gh/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/3468

For the linux documentation, we usually refer to the Wiki. If you remove that I will approve the translation updates and merge.


optimize take and drop

This is not important but for some reason, the use of the imported replicateM_ for take and drop impedes optimization.  I noticed this fiddling with some machines benchmarks of individual functions. See https://gist.github.com/michaelt/f19bef01423b17f29ffd The first list of results is hackage-pipes; the second is with these two repairs
benchmarking drop/conduit
 time                 26.56 ms   (26.31 ms .. 26.79 ms)
benchmarking drop/pipes
 time                 43.64 ms   (42.48 ms .. 45.46 ms)

-- and with the modified drop:
benchmarking drop/pipes
time                 28.03 ms   (26.75 ms .. 29.67 ms)

Then
benchmarking take/conduit
time                 45.38 ms   (44.57 ms .. 46.45 ms)
benchmarking take/pipes
time                 84.44 ms   (78.23 ms .. 91.36 ms)

and with the modified take:
benchmarking take/pipes
time                 55.05 ms   (53.84 ms .. 56.26 ms)

Thank you!  I'm wonder why replicateM_ incurs such a performance hit


Fix the changelog for the needless content

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
API changes are not detected in this pull request.


[Turbo] Passing turbo_stream_listen with components

Right now turbo_stream_listen returns a string.  The StimulusAttributes is casted to a string (it's actually a DTO with toArray function too).
This make impossibile to use it with custom components because we can't pass a string representing the attribute set:
<twig:Code id="attendance" {{ ...turbo_stream_listen("attendance") }}>
</twig:Code>

Is there any solution or workaround to pass the attribute set?
EDIT: I can only think to this to make it DRY. Any better solution?
{# Code.html.twig #}
<div
    {{ attributes
        .defaults({
            class: "mockup-code",
        })
    }}
    {% block attributes %}{% endblock %}
>
    {% block content %}{% endblock %}
</div>

{# StreamedCode.html.twig #}
{% props topic %}

<twig:Code {{ ...attributes }}>
    <twig:block name="attributes">
        {{- turbo_stream_listen(topic) -}}
    </twig:block>

    {{ block(outerBlocks.content) }}
</twig:Code>

Maybe you can pass "attendance" in your twig:Code element and call the turbo stream listen here ?
To pass a stimulus attribute we would need to create another interface / twig function / tests / etc...
I'm not sure this is a very common need, especially compared to the number of people who will help maintain / document / etc afterward :)
But i'm ready to change my mind :)

Yes, @smnandre, I can pass a "topic" prop and then conditionally call turbo_stream_listen inside Code.html.twig. Is that what you meant?
However, it feels wrong to me to "tie" my component to Turbo. It's similar to the way the stimulus_controller function works—I don't want it embedded within my component. I'd prefer to call it externally. Do you agree?
Anyway, if this sounds like an odd request, feel free to close the issue!

Nope, it's not odd, i get what you want here.
The problem is that Turbo was implemented before the HTML Syntax, and its interface prevent to pass attributes like this...
And i guess you don't want to wrap your content ?
<twig:Code id="attendance">
    <div  {{ ...turbo_stream_listen("attendance") }}>

    </div>
</twig:Code>

Yes bu sometimes you cannot, depending on the css framework for example.
Btw, no problem @smnandre this was just an input. Closing this issue!


Allow umlaut domains for website addresses

Checklist

[x] All new and existing tests are passing
[x] (If adding features:) I have added tests to cover my changes
[x] I have added an entry to CHANGES.rst because this is a user-facing change or an important bugfix
[x] I have written proper commit message(s)

What changes does this Pull Request introduce?
Changed website validation to allow domain names containing umlauts
Why is this necessary?
Resolves issue #951

Thank you for this PR. In my eyes, we should avoid introducing new dependencies.
The relevant part of the validators package is domain.py.
The package is MIT licensed and we can copy over that function into isso rather than having to load a whole package for a simple regex check.

Do we actually need to check the domain name at all?

Do we actually need to check the domain name at all?

Because IIRC the website is inserted as a link (if given), we should make sure it is valid.
If we skipped the validity check, I'm not sure that the markup escaping would catch e.g. someone entering malicious Javascript foo.

Do we actually need to check the domain name at all?

Because IIRC the website is inserted as a link (if given), we should make sure it is valid.
If we skipped the validity check, I'm not sure that the markup escaping would catch e.g. someone entering malicious Javascript foo.

That seems like an argument for just fixing the markup escaping to me...

Do we actually need to check the domain name at all?

Because IIRC the website is inserted as a link (if given), we should make sure it is valid.
If we skipped the validity check, I'm not sure that the markup escaping would catch e.g. someone entering malicious Javascript foo.

So, the goal is to actually check the complete entered URL, not only if the domain is valid, as I assumed?

Sorry for the delay; let's just merge this, since it's clearly an improvement over the current situation. Ideally, we'd be moving away from checking for valid domains to checking for malicious things though.


Can ngReact be used with react-bootstrap?

I like ngReact. Great job! Can you please let me know if ngReact can be used with react-bootstrap? I was using angular-bootstrap but since my views are now rendered using React, I can't use the angular-bootstrap anymore. Thank you
http://react-bootstrap.github.io/index.html

@ksmirnoff thanks.
I haven't tried using react-bootstrap with ngReact, so I'll assume it works :wink: . Have you encountered any concrete issues with using react-bootstrap. If so we'll be interested in getting those fixed.

I've begun using react-bootstrap with ngReact and haven't encountered any issues yet.  Granted, the project I'm using them on is still in a very early stage, but so far so good.  I use ngReact's reactDirective service.  Here's an example:
// layout.header.jsx
import React from "react";
import {Affix, Navbar, Nav, NavItem} from "react-bootstrap";
const logo = require("../path/to/logo.png");

class HeaderViewComponent extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      active: props.initialActive,
      userFullName: props.userFullName
    };
  }
  handleSelect(selectedKey) {
    this.setState({active: selectedKey});
  }
  render() {
    return (

        <Navbar fixedTop className="top-global-header">

            <NavItem>{this.state.userFullName}</NavItem>

        </Navbar>
        <Affix offsetTop={36}>
          <Navbar className="top-global-nav">

              <NavItem eventKey={1} href="/dashboard">Dashboard</NavItem>
              <NavItem eventKey={2} href="/events">Events</NavItem>

          </Navbar>
        </Affix>

    );
  }
}
HeaderViewComponent.propTypes = {
  initialActive: React.PropTypes.number,
  userFullName: React.PropTypes.string.isRequired
};
HeaderViewComponent.defaultProps = {initialActive: 1};

export default HeaderViewComponent;

Then in my angular module:
import angular from "angular";
import "ngReact";

import HeaderViewComponent from "./view_components/layout.header";

export default angular.module("app.layout", [..., "react"])
.config(states)
.directive("headerLayout", (reactDirective) => {
  return reactDirective(HeaderViewComponent);
})....blah, blah, other directives or controllers.

From there, if you're using angular-ui-router, you can either just inline the resulting directive markup in your state's template property like I did, or include in your controller's existing html template.
Either way, the resulting directive would look like this:

@jjstrat3 thanks a lot for that detailed reply. I take that as a sign that this issue can be closed.


Reviewed GPU Final Script

Hey @adamkritz, just looked at the GPU final script and thanks for doing all that stuff, it looks good!
Up to you but feel free to remove this random model part if you want as that's not needed for training.
https://github.com/justjoshtings/ms.pacman.ai/blob/112aff81b795eddafcabe4cb74b170b72370cde4/GPU script/Final GPU Script.py#L62-L107
Also don't forget to set nb_steps to something like 600k for the 1mill. This will give it sufficient number of steps to got from 1.0 eps to 0.1 eps.
https://github.com/justjoshtings/ms.pacman.ai/blob/112aff81b795eddafcabe4cb74b170b72370cde4/GPU script/Final GPU Script.py#L196
Also, let's try memory limit to be 500k cause why not lol.
https://github.com/justjoshtings/ms.pacman.ai/blob/112aff81b795eddafcabe4cb74b170b72370cde4/GPU script/Final GPU Script.py#L202

Done.


wip

Fixes #

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: MonthaIRPCaccount sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


feat(Topology/Relative): relative open/closed sets

As suggested by Floris van Doorn in #16710.

Oh, I completely forgot we have this nice s ↓∩ t notation nowadays.
Now I'm not so sure anymore whether IsOpenIn s t is necessary, it's about as long to write as IsOpen (s ↓∩ t). Honestly, in the other PR we could just write IsOpen (S ↓∩ Iio o). I'm on the fence though.
Maybe you could start a thread on Zulip (in #mathlib4)? If we decide to go with this, we need to swap the arguments (IsOpenIn s t should mean that s is open in t, not the other way around).
Pros of IsOpenIn:

will be used quite frequently
new abstraction might suggest some useful new lemmas that are clunkier to write with IsOpen
similarly: could allow nicer dot notation

Cons:

Yet another definition where we need a lot of API / more duplication
It doesn't actually shorten the expression

There is a significant majority on Zulip that prefers to not add this notion.
I think it's still valuable to add a bunch of rewrite rules for ↓∩ that you have here as part of your proof (also the ones that are provable by rfl). I'm also happy to add lemmas like IsOpenIn_sUnion, but with a different name and using the ↓∩ notation.
Feel free to adapt this PR or create a new one (or to let someone else do it)

I think the consensus was not to have these.


astar_search!

Bug report
Required information:

Operating system and version:

Autoware installation type:

< From source, -->

Autoware version or commit hash

ROS distribution and version:

ROS installation type:

Package or library, if applicable:

Description of the bug

[ WARN] [1606979485.298915219]: Invalid start pose
Steps to reproduce the bug

i have nake some try to find the error position,it is in the astar_search.cpp file,and the function of
bool AstarSearch::detectCollision(const SimpleNode& sn)
{
// Define the robot as rectangle
static double left = -1.0 * robot_base2back_;
static double right = robot_length_ - robot_base2back_;
static double top = robot_width_ / 2.0;
static double bottom = -1.0 * robot_width_ / 2.0;
static double resolution = costmap_.info.resolution;
// Coordinate of base_link in OccupancyGrid frame
static double one_angle_range = 2.0 * M_PI / theta_size_;
double base_x = sn.index_x * resolution;
double base_y = sn.index_y * resolution;
double base_theta = sn.index_theta * one_angle_range;
// Calculate cos and sin in advance
double cos_theta = std::cos(base_theta);
double sin_theta = std::sin(base_theta);
// Convert each point to index and check if the node is Obstacle
for (double x = left; x < right; x += resolution)
{
for (double y = top; y > bottom; y -= resolution)
{
// 2D point rotation
int index_x = (x * cos_theta - y * sin_theta + base_x) / resolution;
int index_y = (x * sin_theta + y * cos_theta + base_y) / resolution;
  if (isOutOfRange(index_x, index_y))
  {
    return true;
  }
  else if (nodes_[index_y][index_x][0].status == STATUS::OBS)
  {
    return true;
  }
}

}
return false;
}
i donot why it can detect the collision,there is nothing in the map! anyone can help me? thanks!!!
Expected behavior

the rviz should show a path
Actual behavior

there is nothing
Screenshots

Additional information

@rosliwanlei Please ask support-related questions (including unconfirmed bugs) at https://answers.ros.org/ with the tag autoware.


no source code in container after odo link (odo push is not helping)

▶ odo version
odo v2.2.1 (f9449083e)

Server: https://api.crc.testing:6443
Kubernetes: v1.20.0+7d0a2b2

Installed operators:

"PostgreSQL Operator by Dev4Ddevs.com" 0.1.1
"Service Binding" 0.7.1

▶ git clone https://github.com/spring-projects/spring-petclinic

▶ odo create java-maven

▶ odo config set Memory 1Gi

▶ cat db.yaml
apiVersion: postgresql.dev4devs.com/v1alpha1
kind: Database
metadata:
  name: postgresql
annotations:
  "service.binding/database": "path={.spec.databaseName}"
  "service.binding/username": "path={.spec.databaseUser}"
  "service.binding/password": "path={.spec.databasePassword}"
spec:
  databaseName: example
  databasePassword: postgres
  databaseStorageRequest: 1Gi
  databaseUser: postgres
  image: centos/postgresql-96-centos7
  size: 1

▶ odo service create --from-file db.yaml

#  push can take a long time
▶ odo push --show-log

▶ curl --head http://http-8080-java-maven-spring-petcl-ddcj-tkral-test.apps-crc.testing/
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Content-Language: en
Content-Length: 3161
Date: Wed, 02 Jun 2021 13:56:25 GMT
Set-Cookie: 3b7e1165bfaf6b2b1c0bed3e97f59d2f=24ebcaffe05531acc331c8abf0e28894; path=/; HttpOnly
Cache-control: private

Application is working and responding with 200
▶ odo link Database/postgresql

# wait for pod to restart and be in Running state again
▶ k get pods -w
NAME                                           READY   STATUS    RESTARTS   AGE
java-maven-spring-petcl-ddcj-58db4b8b8-jgls7   1/1     Running   0          15s
postgresql-85c4b4488b-pfll5                    1/1     Running   1          15m
postgresql-operator-6b67c45dfc-2vrcs           1/1     Running   0          15m

Application is not running, getting HTTP error 500
▶ curl --head http://http-8080-java-maven-spring-petcl-ddcj-tkral-test.apps-crc.testing/
HTTP/1.0 503 Service Unavailable
Pragma: no-cache
Cache-Control: private, max-age=0, no-cache, no-store
Connection: close
Content-Type: text/html

This is becuase there is no source code in the container
▶ k exec -it java-maven-spring-petcl-ddcj-58db4b8b8-jgls7 /bin/bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "tools" out of: tools, copy-supervisord (init)
I have no name!@java-maven-spring-petcl-ddcj-58db4b8b8-jgls7:/projects$ ls -lah
total 0
drwxrwsrwx. 2 root<PHONE_NUMBER>  6 Jun  2 13:58 .
dr-xr-xr-x. 1 root root       51 Jun  2 13:58 ..

If I try to run odo push -f to force push files it is not syncing any files.
▶ odo push -f
 ✓  Waiting for component to start [8ms]

Validation
 ✓  Validating the devfile [52090ns]

Creating Kubernetes resources for component java-maven-spring-petcl-ddcj
 ✓  Waiting for component to start [1ms]
 ✓  Waiting for component to start [1ms]

Applying URL changes
W0602 15:59:44.794071  326590 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
 ✓  URLs are synced with the cluster, no changes are required.

Syncing to component java-maven-spring-petcl-ddcj
 ✓  Checking file changes for pushing [15ms]
 ✓  Syncing files to the component [225ms]

Executing devfile commands for component java-maven-spring-petcl-ddcj
 ✓  Waiting for component to start [2ms]
 ✗  Executing mvn-package command "mvn -Dmaven.repo.local=/home/user/.m2/repository package" [975ms]
 ✗  Failed to start component with name "java-maven-spring-petcl-ddcj". Error: Failed to create the component: command execution failed: unable to execute the run command: unable to exec command [/bin/sh -c mvn -Dmaven.repo.local=/home/user/.m2/repository package]:
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.061 s
[INFO] Finished at: 2021-06-02T13:59:45Z
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/projects). Please verify you invoked Maven from the correct directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
: error while streaming command: command terminated with exit code 1

/kind bug
because result is not working application
/priority high

I couldn't find a test that would verify that the application is still running after the link is created!

This would be related to source code not being present in the pvc mount I suppose?

The reason is that odo link only creates a servicebinding, and the effect is that the SBO creates a new pod to make the link effective.
But odo didn't have a chance to re-synchronize the file into the /project directory after SBO recreates the pod.
As the flow changes with https://github.com/openshift/odo/issues/4208, it should fix this issue.

I couldn't find a test that would verify that the application is still running after the link is created!

@feloy can we also ensure that we have an integration test for this?

/unassign
/assign @feloy


Fix XYChart area fill for negative values

Closes #326
Previously filled between values and bottom of the chart, not values and 0.

This PR broke normal chart rendering. Please submit a fix including a sample chart like others found in the standalone/issues package.

Or add a AreaChart05 to the XChart Demo.


When you set on an empty path, the zookeeper will crash

When you try setting data on an empty path, it will crash and burn your zookeeper server. And by this I mean data corruption, you need to empty your data directory and then restart it, otherwise nobody will be able to connect to it any more and will get a connection refused.
It would be nice to have an error if the path is empty, rather than letting the set happen and watching your zookeeper go down in style.

Hi,
Can you give more details:

Which version you are using
The code snip that caused this
Any suspicious logs?

Thanks!

This is a bug in zookeeper, see https://issues.apache.org/jira/browse/ZOOKEEPER-2213

Thanks for reporting this.
While it certainly seems like the server should handle this more gracefully, I added a client side check to be safe.

I create a ticket to have the logic in this package match the Java client: https://github.com/samuel/go-zookeeper/issues/63

Thanks guys, damn you work really fast :+1:


🚀 : Update birthdate and name (only for the first time)

Is your feature request related to a problem? Please describe.
I'm always frustrated when I can't update my birthdate and only have an unlimited ability to update my name, which can lead to potential inconsistencies in my account information.
Describe the solution you'd like
I would like an option to update my birthdate only once after the initial setup, to correct any errors made during registration. Additionally, I would like the ability to update my name, but with a limit of only one time per account.
Describe alternatives you've considered
An alternative could be to allow users to request changes to their birthdate and name through customer support. However, this process is less convenient and slower compared to allowing controlled updates directly within the app.
Additional context
Allowing a one-time update for both birthdates and names will enhance user experience by providing necessary flexibility while maintaining data integrity. Here's a screenshot of the current settings page for reference.
Contributor: @carrot from Discord
If you want to speed up the process of this issue, please send 🚀 as a reaction.

There's no need to address this issue anymore; we only need to focus on issue #776. Regarding the birthdate functionality, it should be included in the support mail or discord support.


Callback after graph is rendered?

I may be missing something, but it would be awesome if I could create a callback after a graph is rendered. I'm attempting to provide a screenshot to the user after calling ZoomToFit, but the ZoomToFit function returns before the graph has been re-rendered, so the screenshot is of the graph's previous state.

Hi @destrugter,
I see how that would be useful, but am not sure whether it's going to be easy to implement. It might be as easy as providing a callback/event in OnAfterRenderAsync of the Diagram component, but I don't know for sure. Could you experiment with this and let me know?
Out of curiosity: how are you taking screenshots?
BR
Stefan

Currently the way I'm doing it is calling ZoomToFit, awaiting a 1s Task, then using html2canvas to convert the element to a canvas, and then converting it to a blob, and finally converting that blob to a png and copying it to the clipboard.

@CliveBennett I used this JS called html2canvas. It works pretty well. The command I have after you have downloaded html2canvas is await html2canvas(document.querySelector("#" + id)).then(canvas => canvas.toBlob(blob => navigator.clipboard.write([new ClipboardItem({ 'image/png': blob })])));

Awesome, appreciate that ill give it a go.

I managed to get the copying of the diagram to a PNG working, thanks for the pointer.
But, because some of the diagrams that get generated can be quite large, I really need to have a dynamic sized image, so I can have a large image for the large ones, and a small image for the small ones.
Is there a property for the diagram class that would give me the max rendered width and height in pixels?, even if cropped on screen?

@CliveBennett The code I was using was this:
private async Task GetDiagramScreenshot()
{
	diagram.ZoomToFit();
	await Task.Delay(1000);
	image_url = await TakeScreenshot();
}

I am using the await Task.Delay(1000) because I am unsure of when ZoomToFit is finished working, which is why I opened this issue in the first place. If I'm understanding you correctly, this should get you what you're after?

@destrugter the issue is that a lot of our diagrams can render off screen as they are quite large, I use the little viewport control to enable people to scroll around the diagram.
Our webui looks like this

But I needed to grab the entire diagram in one hit onto a single image.
What I ended up doing was to change the containing div to 5000x5000 pix with JS prior to running Html2Canvas which then generates a large image with the diagram at the top, then I wrote some code to Crop the image to a sensible size by scanning the pixels for whitespace.
Works well (so far!) thanks again for the help


Subset sum possibility program

What does this PR do?
This PR consists of code to find the possibility of obtaining a number from the sum of the subset. Method subsetSumPossible() takes an integer array and an integer (sum) as a parameter and checks if the sum can be obtained from the subset of the array. Returns true if it's possible to obtain the @param sum from the subset or else returns false.
Test Plan
No
Related PRs and Issues
Related to issue #575
Have you read the Contributing Guidelines?
Yes

Hi @dubesar Please add hactoberfest-accepted label this PR.


✨ Built-in RTMP Support in React Native Vision Camera

What feature or enhancement are you suggesting?
I'm suggesting the addition of RTMP (Real-Time Messaging Protocol) streaming support in the React Native Vision Camera library. This feature would allow developers to stream live video and audio directly from the camera to an RTMP server, facilitating the creation of live streaming applications within the React Native ecosystem.
What Platforms whould this feature/enhancement affect?
iOS, Android
Alternatives/Workarounds
Current Situation

We currently need custom native modules or third-party libraries for RTMP streaming.
use libraries like React-Native-WebRTC, FFmpeg Kit or external RTMP SDKs
This requires significant development effort:

Managing video/audio stream encoding
Handling RTMP protocol complexities

Existing Libraries

React-Native-NodeMediaClient
NodeMedia-Client-with-Zoom
React-Native-RTMP-Publisher

Issues with existing libraries:

First two are unmaintained and outdated
RTMP Publisher doesn't support the new React Native architecture

Request
Could we consider adding built-in RTMP support to React Native Vision Camera?
Take RTMP link as output URL.
Benefits:

Ensure compatibility with the latest React Native architecture

Questions

Is this feature on the roadmap?
What challenges do you foresee in implementing this?
How can the community contribute to making this happen?

Thank you for considering this request and for your ongoing work on React Native Vision Camera!
You guys are doing great in React Native Community✨.
Additional information

[X] I agree to follow this project's Code of Conduct
[X] I searched for similar feature requests in this repository and found none.

Hey!
Thanks for the feature request, but this is not something I will implement for VisionCamera.
This is exactly the reason why I built Frame Processors, a modular system allowing you to do ANYTHING with video frames, in a powerful and flexible way. There can be a WebRTC plugin that users can simply use via JS (that's the easy-to-use part), and then you integrate this.
We have already built a few such tailored solutions for a few clients before.

How can the community contribute to making this happen?

By building a VisionCamera WebRTC frame processor plugin 😄

I appreciate the direction towards using Frame Processors for this purpose. Since RTMP streaming is a bit complex, I’m eager to contribute by building the Vision Camera WebRTC frame processor plugin myself with help from others in the community.
However, I’m not entirely sure where to start since I don't have much experience with TurboModules and C++. If you could spare some time, could you guide me on the key steps involved in creating such a plugin?
Specifically, I’m looking for advice on:

Integrating WebRTC: How would you recommend starting the integration with WebRTC for handling video frames from Vision Camera?
Frame Processing Basics: Any tips on the essentials of creating a robust frame processor plugin for live streaming?
Key Challenges: What are the potential pitfalls or challenges I should be aware of while developing this?

If you have any resources or pointers to relevant parts of the Vision Camera codebase that could help, I’d be grateful.
Thanks again for your time and guidance!

You don't need C++, nor TurboModules.
Check out the docs here on FP Plugins


Extract required data via Buildkite API

In order to surface the information we need we need to define what this information is and where and how we can best source it.


Project description
NordVPN's native app is the only way to access some of their VPN features. They provide .deb and .rpm packages, which are compatible with various Debian-based and RHEL-based systems, however I don't know enough about nix/nixpkgs to know whether we can use these in NixOS.
Metadata

homepage URL: https://nordvpn.com/
source URL: https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn-release_1.0.0_all.deb
license: unfree
platforms: linux

I have no idea what I am doing but following a blog post over at reflexivereflection.com and the nixos wiki about deb packages in nixos lead me to nordvpn.nix and builder.sh
nix-env -f nordvpn.nix nordvpn
sudo nordvpnd # this complains about public keys
nordvpn # prints help, anything else does not work due to nordvpnd not running

I was able to run the nordvpn service by running it in a bubblewrap environment... This makes nordvpnd find the stuff it expects under /var/lib/nordvpn/... unfortunately the service wants to write into /var/lib/nordvpn/data/settings.dat and because that is mounted as read only it fails
2020/11/08 19:03:39 [Error] open /var/lib/nordvpn/data/settings.dat: read-only file system
2020/11/08 19:03:39 [Error] open /var/lib/nordvpn/data/settings.dat: no such file or directory

Here is the code. nordvpn is here just the definition from @jakubdaniel
pkgs.buildFHSUserEnvBubblewrap {
  name = "nordvpnd";
  targetPkgs = pkgs: [ nordvpn ];
  runScript = "${nordvpn}/usr/sbin/nordvpnd";
}

Maybe anyone else has a tip how to circumvent this issue?

I have been playing around with this for the last day and it seems to be a very interesting thing to try and package for nix. The .deb only adds a GPG key and the location of the nordvpn repository to your apt package manager, so you can then download and update the package through apt normally. This means that one cannot use the same style of packaging as we see in other .deb packages like opera, skype, or teams.

Currently trying to see if there is a compiled binary somewhere on their repository and try packaging that directly, but probably will run into an issue of needing the GPG key.

Disclaimer: I've only recently started using nix. I've been working on this alittle bit and I think I've narrowed down exactly where the issue lies. FirstI've updated the nordvpn version to the latest because it will complain if itisn't the latest. Here is my first file nordvpnd.nix that combines all thestuff from this thread so far:
{ pkgs ? import <nixpkgs> {} }:

with pkgs;
let
  nordvpn = stdenv.mkDerivation {
      name = "nordvpn";

      src = fetchurl {
        url = "https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn_3.8.8_amd64.deb";
        sha256 = "0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm";
      };

      nativeBuildInputs = [ dpkg ];

      phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

      unpackPhase = "dpkg -x $src unpacked";

      installPhase = ''
        mkdir -p $out/usr
        cp -r unpacked/* $out/
        patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/bin/nordvpn
        patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/sbin/nordvpnd
      '';

      dontPatchShebangs = true;
      dontStrip = true;
      dontPatchELF = true;
      dontAutoPatchelf = true;

  };
in pkgs.buildFHSUserEnvBubblewrap {
  name = "nordvpnd";
  targetPkgs = pkgs: [ nordvpn ];
  runScript = "${nordvpn}/usr/sbin/nordvpnd";
}

So if you take that file and place it somewhere and execute nix-build nordvpnd.nix as root and then run ./result/bin/nordvpnd you should see the error first:
Error on listening to UNIX domain socket: listen unix /run/nordvpn/nordvpnd.sock: bind: no such file or directory`

It looks like they've updated the located of the socket so you will have to mkdir /run/nordvpn and rerun again to get:
open /var/lib/nordvpn/data/settings.dat: read-only file system

If you run it mutliple times you might also run into:
Error on listening to UNIX domain socket: listen unix /run/nordvpn/nordvpnd.sock: bind: address already in use

in which case you can you can just remove /run/nordvpn/nordvpnd.sock and run it again. Ok, now that we have everything set up here is where this becomes an issue. When you unpack nordvpn with dpkg -x $src unpacked you can see that it contains 3 folders etc, usr, and var. Now if you look at the pkgs/build-support/build-fhs-userenv-bubblewrap/default.nix for the buildFHSUserEnvBubblewrap command you will notice this script in the bwrapCmd:
for i in ${env}/*; do
  path="/''${i##*/}"
  if [[ $path == '/etc' ]]; then
    continue
  fi
  echo $i $path
  ro_mounts+=(--ro-bind "$i" "$path")
  blacklist+=("$path")
done

Where env is the bubble wrapped filesystem. Essentially this means that any folder that is created by the package, in our case we create a /var/. will be marked as readonly and therefore can't be written to. So unfortunately seems like and edge case that isn't handled with BubbleWrap so I don't think that's going to work. If we can't bubble wrap though it is possible to just manually move the files that are required onto our "host" system. Consider the following file which is essentially the above one without the bubblewrap (I called this one nordvpn.nix)
{ pkgs ? import <nixpkgs> {} }:

with pkgs;
stdenv.mkDerivation {
    name = "nordvpn";

    src = fetchurl {
      url = "https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn_3.8.8_amd64.deb";
      sha256 = "0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm";
    };

    nativeBuildInputs = [ dpkg ];

    phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

    unpackPhase = "dpkg -x $src unpacked";

    installPhase = ''
      mkdir -p $out/usr
      cp -r unpacked/* $out/
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/bin/nordvpn
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/sbin/nordvpnd
    '';

}

After that you can run nix-build nordvpn.nix and you will again get your result folder that has all the required files. You can execute ./result/usr/sbin/nordvpnd to get the daemon running and then if you get complaints such as:
retrieving public key from vault: error reading file: open /var/lib/nordvpn/data//rsa-key-1.pub: no such file or directory

You can just copy the file manually like so:
cp result/var/lib/nordvpn/data/rsa-key-1.pub /var/lib/nordvpn/data/

and then re-run the ./result/usr/sbin/nordvpnd command. Or just copy the entire result/var/lib/nordvpn over. I've got nordvpnd running but I've still not been able to get is to connect, although it does allow me to login to my account. If I run ./result/usr/sbin/nordvpnd in one terminal and try to connect with ./result/usr/bin/nordvpn connect in another one the daemon will print out all of the commands but in the client window it says it fails. This part is starting to get beyond me so I'll probably give up here.  I've spent quite a bit of time tinkering and hopefully this will help someone else who's trying to take a full stab at this problem better understand the issue.

I'm still new to NixOS, so I don't know how to go about implementing this..
I think I understand what needs to be done:

Debian package has to be downloaded.
It's contents need to be extracted into the output directory.
ELF needs to be patched to not rely on FHS.
SystemD files need to be included from the original package.
Sandboxing the app is not going to work, because it needs read/write access to some parts of the system, which are global, for example, firewall.

I'm still not sure how to test this derivation locally..
What I've done so far is sudo nix-env -i -f default.nix, but it does not place binaries in the path. SystemD files are also not detected.. Therefore I'm unable to see if this actually works before submitting code changes.
I've tried linking service and socket files manually, but it seems that I'm unable to do so..
sudo systemctl link result/usr/lib/systemd/system/*
Failed to link unit: File /etc/systemd/system/nordvpnd.service: Read-only file system

Here is how my current derivation looks like.
cat default.nix

{ pkgs ? import <nixpkgs> {} }:

with pkgs;

stdenv.mkDerivation {
    name = "nordvpn";

    src = fetchurl {
        url = "https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn_3.8.9_amd64.deb";
        sha256 = "7af59ef35c0f9c6bda3ad8cab1d962b340a7a030725e857413b28f9458c950b7";
    };

    nativeBuildInputs = [ dpkg ];

    phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

    unpackPhase = "dpkg -x $src unpacked";

    installPhase = ''
        mkdir -p $out/usr
        cp -r unpacked/* $out/
        patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/bin/nordvpn
        patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/sbin/nordvpnd
    '';

    dontPatchShebangs = true;
    dontStrip = true;
    dontPatchELF = true;
    dontAutoPatchelf = true;
}

Today I've tried following Nixpkgs documentation: https://nixos.org/manual/nixpkgs/stable/#submitting-changes-making-patches
My fork of Nixpkgs: https://github.com/domust/nixpkgs/commit/b92e6875c33b6ff9d217c8cb23be33a0efddad70
I've added the following lines to my /etc/nixos/configuration.nix:
  nixpkgs.config.allowUnfree = true;
  environment.systemPackages = with pkgs; [
    nordvpn
  ];

and executed the following command:
sudo nixos-rebuild switch -I /home/-/devel/nixpkgs
building Nix...
building the system configuration...
error: undefined variable 'nordvpn' at /etc/nixos/configuration.nix:52:5
(use '--show-trace' to show detailed location information)

Am I missing something?

Today I've tried following Nixpkgs documentation: https://nixos.org/manual/nixpkgs/stable/#submitting-changes-making-patches
My fork of Nixpkgs: https://github.com/domust/nixpkgs/commit/b92e6875c33b6ff9d217c8cb23be33a0efddad70
I've added the following lines to my /etc/nixos/configuration.nix:
  nixpkgs.config.allowUnfree = true;
  environment.systemPackages = with pkgs; [
    nordvpn
  ];

and executed the following command:
sudo nixos-rebuild switch -I /home/-/devel/nixpkgs
building Nix...
building the system configuration...
error: undefined variable 'nordvpn' at /etc/nixos/configuration.nix:52:5
(use '--show-trace' to show detailed location information)

Am I missing something?

I think I have made significant progress today. I needed to define service module, which handles systemd services.
My code so far: https://github.com/domust/nixpkgs/commit/1ccd3e1928e89bb313dfbf75eea49c7b1c7c26c0

I think I have made significant progress today. I needed to define service module, which handles systemd services.
My code so far: https://github.com/domust/nixpkgs/commit/1ccd3e1928e89bb313dfbf75eea49c7b1c7c26c0

I managed to make some progress using your fork as an overlay with some additional modifications:

Add to installPhase:

mv $out/usr/* $out
rmdir $out/usr

Remove accordingly the usr/s in fixupPhase
Add a nordvpn group as referenced in the socket

users.groups.nordvpn = {};

The first two are due to Nix in general (as well as your service) assuming that bin/ and sbin/ belong in $out rather than $out/usr.
With these modifications, the CLI tool and service run, but the next obstacle to solve would be nordvpnd looking for the files in /var/lib/nordvpn, which fails as they are located in /nix/store/${hash}-nordvpn-3.8.9/var/lib/nordvpn instead—I'll check back in if I make progress on that front.

I managed to make some progress using your fork as an overlay with some additional modifications:

Add to installPhase:

mv $out/usr/* $out
rmdir $out/usr

Remove accordingly the usr/s in fixupPhase
Add a nordvpn group as referenced in the socket

users.groups.nordvpn = {};

The first two are due to Nix in general (as well as your service) assuming that bin/ and sbin/ belong in $out rather than $out/usr.
With these modifications, the CLI tool and service run, but the next obstacle to solve would be nordvpnd looking for the files in /var/lib/nordvpn, which fails as they are located in /nix/store/${hash}-nordvpn-3.8.9/var/lib/nordvpn instead—I'll check back in if I make progress on that front.

@domust you can replace both the systemd service and socket declaration with systemd.packages = [ pkgs.nordvpn ]; because the package already ships these files :+1: Just make sure you patch any incorrect absolute paths in the nordvpn derivation.

@domust you can replace both the systemd service and socket declaration with systemd.packages = [ pkgs.nordvpn ]; because the package already ships these files :+1: Just make sure you patch any incorrect absolute paths in the nordvpn derivation.

I have made changes according to @chuahou and @aanderse comments. The resulting code is here: https://github.com/domust/nixpkgs/commit/4a0edd7145826d5fdeea6aa60c63c1368b6e03c4
But unlike @chuahou, I'm still unable to run the daemon and verify my changes. I'm testing them using build-vm command.
VM configuration is i a file named vmtest.nix with the following contents:
{config, pkgs, ...}:

{
  # You need to configure a root filesytem
  fileSystems."/".label = "vmdisk";

  # The test vm name is based on the hostname, so it's nice to set one
  networking.hostName = "vmhost";

  # Add nordvpn group
  users.groups.nordvpn = {};

  # Add a test user who can sudo to the root account for debugging
  users.extraUsers.vm = {
    password = "vm";
    shell = "${pkgs.bash}/bin/bash";
    group = "wheel";
    extraGroups = [ "nordvpn" ];
  };
  security.sudo = {
    enable = true;
    wheelNeedsPassword = false;
  };

  nixpkgs.config.allowUnfree = true;
  # Enable your new service!
  services.nordvpn.enable = true;
}

I'm building vm with the following command while at the root of nixpkgs repository:
NIXOS_CONFIG=pwd/vmtest.nix nixos-rebuild -I nixpkgs=pwd -I nixos=pwd/nixos/ build-vm
When I run a resulting vm with ./result/bin/run-vmhost-vm and login into vm user, I cannot find nordvpn in path.
systemctl status nordvpnd reports service not found.

I have made changes according to @chuahou and @aanderse comments. The resulting code is here: https://github.com/domust/nixpkgs/commit/4a0edd7145826d5fdeea6aa60c63c1368b6e03c4
But unlike @chuahou, I'm still unable to run the daemon and verify my changes. I'm testing them using build-vm command.
VM configuration is i a file named vmtest.nix with the following contents:
{config, pkgs, ...}:

{
  # You need to configure a root filesytem
  fileSystems."/".label = "vmdisk";

  # The test vm name is based on the hostname, so it's nice to set one
  networking.hostName = "vmhost";

  # Add nordvpn group
  users.groups.nordvpn = {};

  # Add a test user who can sudo to the root account for debugging
  users.extraUsers.vm = {
    password = "vm";
    shell = "${pkgs.bash}/bin/bash";
    group = "wheel";
    extraGroups = [ "nordvpn" ];
  };
  security.sudo = {
    enable = true;
    wheelNeedsPassword = false;
  };

  nixpkgs.config.allowUnfree = true;
  # Enable your new service!
  services.nordvpn.enable = true;
}

I'm building vm with the following command while at the root of nixpkgs repository:
NIXOS_CONFIG=pwd/vmtest.nix nixos-rebuild -I nixpkgs=pwd -I nixos=pwd/nixos/ build-vm
When I run a resulting vm with ./result/bin/run-vmhost-vm and login into vm user, I cannot find nordvpn in path.
systemctl status nordvpnd reports service not found.

@domust I'm not overly familiar with the correct way in NixOS yet, but I believe you need to also have environment.systemPackages = [ pkgs.nordvpn ]; to have the package in your path, as systemd.packages will only handle the systemd units. If you do that you should be able to run nordvpn.
Also, the systemd units are nordvpn.service and nordvpn.socket (without the d), which is why systemctl status nordvpnd would report "not found".
Lastly, I should probably clarify that I managed to get the service and CLI tool to run (start and produce output), but not to run correctly, as they still produce errors I unfortunately haven't been able to try fixing yet.

@domust I'm not overly familiar with the correct way in NixOS yet, but I believe you need to also have environment.systemPackages = [ pkgs.nordvpn ]; to have the package in your path, as systemd.packages will only handle the systemd units. If you do that you should be able to run nordvpn.
Also, the systemd units are nordvpn.service and nordvpn.socket (without the d), which is why systemctl status nordvpnd would report "not found".
Lastly, I should probably clarify that I managed to get the service and CLI tool to run (start and produce output), but not to run correctly, as they still produce errors I unfortunately haven't been able to try fixing yet.

@chuahou I was under impression that writing service like this allows to have one line install just like in the docker example above, especially when the software is designed to be run as a service and stand alone package is not that that useful on its own.

@chuahou I was under impression that writing service like this allows to have one line install just like in the docker example above, especially when the software is designed to be run as a service and stand alone package is not that that useful on its own.

@domust If I'm not wrong that will install the systemd units and the software the units run (i.e. nordvpnd), but nordvpn is the CLI tool for the user to call, so for that to be added to the path in the user environment you would need environment.systemPackages.
For example, for blueman these lines show adding the user executables, the DBus configuration and the systemd units respectively.

@domust If I'm not wrong that will install the systemd units and the software the units run (i.e. nordvpnd), but nordvpn is the CLI tool for the user to call, so for that to be added to the path in the user environment you would need environment.systemPackages.
For example, for blueman these lines show adding the user executables, the DBus configuration and the systemd units respectively.

It seems that I've hit this issue: https://github.com/NixOS/nixpkgs/issues/97305
That's why I'm having problems booting the vm and the password seems to not work.

It seems that I've hit this issue: https://github.com/NixOS/nixpkgs/issues/97305
That's why I'm having problems booting the vm and the password seems to not work.

I think I know why service is dead. There are two reasons:

Nix expects binaries to be in $package/bin and not $package/sbin.
nordvpnd.service has hardcoded ExecStart=/usr/sbin/nordvpnd.

I know how to solve 1, but not 2. Would it be possible to change the value of ExecStart to nix store path during fixupPhase or is there some other way to solve this problem?

I think I know why service is dead. There are two reasons:

Nix expects binaries to be in $package/bin and not $package/sbin.
nordvpnd.service has hardcoded ExecStart=/usr/sbin/nordvpnd.

I know how to solve 1, but not 2. Would it be possible to change the value of ExecStart to nix store path during fixupPhase or is there some other way to solve this problem?

Right, that is because in order to evaluate drv variable you need to evaluate drv inside the string. Instead of interpolating drv Nix variable, just use $out environment variable. Or interpolate placeholder "out" Nix expression.

Right, that is because in order to evaluate drv variable you need to evaluate drv inside the string. Instead of interpolating drv Nix variable, just use $out environment variable. Or interpolate placeholder "out" Nix expression.

@jtojnar's solution worked. I was able to run the service, but new problems appeared:

Service is not started by default, I've probably missed something in the service configuration.
The daemon tries to access files at /var/lib/nordvpn/data/, but those are at $out/var/lib/nordvpn/data/ and I was thinking if I could do something like patchelf` again.

@jtojnar's solution worked. I was able to run the service, but new problems appeared:

Service is not started by default, I've probably missed something in the service configuration.
The daemon tries to access files at /var/lib/nordvpn/data/, but those are at $out/var/lib/nordvpn/data/ and I was thinking if I could do something like patchelf` again.

I was able to get the code at the end of my last post working by disabling the built in firewall:
networking.firewall.enable = false;

and then adding iptables to my packages. I can only connect with nordlynx I wasn't able to figure out the issue with openvpn.

I was able to get the code at the end of my last post working by disabling the built in firewall:
networking.firewall.enable = false;

and then adding iptables to my packages. I can only connect with nordlynx I wasn't able to figure out the issue with openvpn.

@thornycrackers if I understand correctly, patching ELF is error prone and due to new path being longer than the FHS one, patching would require binary padding. So using mount namespaces with FHS is easier and it just so happens, that bubblewrap has this covered and the tool itself is well supported on NixOS? What I don't understand about bubblewrap is how you've managed to connect, because their README clearly states: https://github.com/containers/bubblewrap#user-namespaces
Bubblewrap could be viewed as setuid implementation of a subset of user namespaces. Emphasis on subset - specifically relevant to the above CVE, bubblewrap does not allow control over iptables.

Iptables should now work at all. :confused:

@thornycrackers if I understand correctly, patching ELF is error prone and due to new path being longer than the FHS one, patching would require binary padding. So using mount namespaces with FHS is easier and it just so happens, that bubblewrap has this covered and the tool itself is well supported on NixOS? What I don't understand about bubblewrap is how you've managed to connect, because their README clearly states: https://github.com/containers/bubblewrap#user-namespaces
Bubblewrap could be viewed as setuid implementation of a subset of user namespaces. Emphasis on subset - specifically relevant to the above CVE, bubblewrap does not allow control over iptables.

Iptables should now work at all. :confused:

The bash script that does the bind mounting won't work in it's current implementation for nordvpn. You can see my post for an explanation. Nordvpn needs write access for /var/lib/nordvpn/data/settings.dat. To give it write access you'd need to use a bind mount to the host system, which won't work because the nordvpn has a var in it's unpackaged directory. I have the following solution to just get nordvpn working, it is ugly but at least I can use the VPN. I ran the following commands as root:

create nordvpn.nix

{ pkgs ? import <nixpkgs> {} }:

with pkgs;
stdenv.mkDerivation {
    name = "nordvpn";

    src = fetchurl {
      url = "https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn_3.8.8_amd64.deb";
      sha256 = "0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm";
    };

    nativeBuildInputs = [ dpkg ];

    phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

    unpackPhase = "dpkg -x $src unpacked";

    installPhase = ''
      mkdir -p $out/usr
      cp -r unpacked/* $out/
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/bin/nordvpn
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/sbin/nordvpnd
    '';

}

Make sure networking.firewall.enable = false; but you have iptables installed. Something with the generated rules messes with nordvpn and I don't know how to figure out exactly what the issue is but I had to do this or when I tried to connect it would just kill my internet connection.
nix-build nordvpn to build the package. For readers unfamiliar this will create the result directory with the files.
cp -r result/var/lib/nordvpn /var/lib/nordvpn I just manually copy these files to my host system.
in terminal 1 run ./result/usr/sbin/nordvpnd to start the daemon. This command might complain about listen unix /run/nordvpn/nordvpnd.sock: in which case you just need to mkdir /run/nordvpn
In terminal 2 run ./result/usr/bin/nordvpn login to get access to your account.
In terminal 2 run ./result/usr/bin/nordvpn set technology nordlynx to set the technology to nordlynx. I've tried adding the patchelf command on the /var/lib/nordvpn/openvpn file but it didn't change the outcome for me. I couldn't get the service to ever start with openvpn only nordlynx.
In terminal 2 run ./result/usr/bin/nordvpn connect

This will get you a Frankenstein version of Nordvpn working, it's ugly but I haven't seen any other way to get NordLynx working on NixOS.

The bash script that does the bind mounting won't work in it's current implementation for nordvpn. You can see my post for an explanation. Nordvpn needs write access for /var/lib/nordvpn/data/settings.dat. To give it write access you'd need to use a bind mount to the host system, which won't work because the nordvpn has a var in it's unpackaged directory. I have the following solution to just get nordvpn working, it is ugly but at least I can use the VPN. I ran the following commands as root:

create nordvpn.nix

{ pkgs ? import <nixpkgs> {} }:

with pkgs;
stdenv.mkDerivation {
    name = "nordvpn";

    src = fetchurl {
      url = "https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn_3.8.8_amd64.deb";
      sha256 = "0v6r3wwnsk5pdjr188nip3pjgn1jrn5pc5ajpcfy6had6b3v4dwm";
    };

    nativeBuildInputs = [ dpkg ];

    phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

    unpackPhase = "dpkg -x $src unpacked";

    installPhase = ''
      mkdir -p $out/usr
      cp -r unpacked/* $out/
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/bin/nordvpn
      patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out/usr/sbin/nordvpnd
    '';

}

Make sure networking.firewall.enable = false; but you have iptables installed. Something with the generated rules messes with nordvpn and I don't know how to figure out exactly what the issue is but I had to do this or when I tried to connect it would just kill my internet connection.
nix-build nordvpn to build the package. For readers unfamiliar this will create the result directory with the files.
cp -r result/var/lib/nordvpn /var/lib/nordvpn I just manually copy these files to my host system.
in terminal 1 run ./result/usr/sbin/nordvpnd to start the daemon. This command might complain about listen unix /run/nordvpn/nordvpnd.sock: in which case you just need to mkdir /run/nordvpn
In terminal 2 run ./result/usr/bin/nordvpn login to get access to your account.
In terminal 2 run ./result/usr/bin/nordvpn set technology nordlynx to set the technology to nordlynx. I've tried adding the patchelf command on the /var/lib/nordvpn/openvpn file but it didn't change the outcome for me. I couldn't get the service to ever start with openvpn only nordlynx.
In terminal 2 run ./result/usr/bin/nordvpn connect

This will get you a Frankenstein version of Nordvpn working, it's ugly but I haven't seen any other way to get NordLynx working on NixOS.

@thornycrackers I think I know how to get around problem number 4. https://www.freedesktop.org/software/systemd/man/systemd.exec.html#WorkingDirectory=
I have found this in gitea's derivation: https://github.com/NixOS/nixpkgs/blob/release-19.03/nixos/modules/services/misc/gitea.nix#L407
I'm still trying this out, though.

@thornycrackers I think I know how to get around problem number 4. https://www.freedesktop.org/software/systemd/man/systemd.exec.html#WorkingDirectory=
I have found this in gitea's derivation: https://github.com/NixOS/nixpkgs/blob/release-19.03/nixos/modules/services/misc/gitea.nix#L407
I'm still trying this out, though.

source URL: https://repo.nordvpn.com/deb/nordvpn/debian/pool/main/nordvpn-release_1.0.0_all.deb
license: unfree

all the mainstream VPN providers expect their users to install CLOSED SOURCE software
which is 100% ridiculous, given that openVPN and wireguard are open source protocols
and the only good reason for closed source is spyware and backdoors
all that the client must do is login and download keys for crypto (two http requests)
which can be implemented in like 100 lines of bash code
i recommend https://www.privateinternetaccess.com/ as VPN provider
they offer multiple open source clients
https://github.com/pia-foss/manual-connections
https://github.com/pia-foss/manual-connections#3rd-party-repositories
https://github.com/pia-foss/desktop
https://github.com/pia-foss/android

so here

will make a draft PR in a bit

Today I found out, that NordVPN actually supports OpenVpn.
It's not as great as the software in terms of automatically selecting a country or a good server for you,
but I don't have to install any additional software which is great.
See this page: https://nordvpn.com/de/ovpn/

Today I found out, that NordVPN actually supports OpenVpn. It's not as great as the software in terms of automatically selecting a country or a good server for you, but I don't have to install any additional software which is great. See this page: https://nordvpn.com/de/ovpn/
Be aware that ipv6 is not supported and thus may leak if you connect via ipv6. I tried to disable ipv6 with networking.enableIPv6 = false, but that lead to all kinds of strange network problems disappointed

could you expand on your comment?
when you say ipv6 is not supported and thus may leak, what do you mean, specifically?
I'm asking because I just migrated into a fresh nixOS setup, and I need to mix a nordvpn and wireguard (ipv6) tunnels.
I'm calculating the best approach for nordvpn, since it seems there's not yet any canonical solution to the problem

@bernardoaraujor as far as I understand it, NordVPN (as most VPN provider) does not offer vpn support for ipv6. This would mean that when your computer determines to use ipv6 for a certain connection (like a tcp connection for a http request/response), this connection would not go through your VPN but just directly. It can chose to use ipv6 in situations where your computer, your isp, the dns server, and the targeted server all support ipv6. You can check your ipv6 status here for example: https://test-ipv6.com/.
Because this would not be great for your anonymity, the NordVPN software "goes the extra mile" and blocks ipv6 (I think directly in the network interface) when the vpn is active. In that way they ensure that your whole traffic goes over ipv4 and thus over the vpn. you can read about their leak protection here.
But if you are just using the normal openvpn software with a specific nordvpn server, no one is blocking your ipv6 interface and thus some connections might chose to bypass the VPN.
To avoid this, I am disabling ipv6 in my wlan connection at the moment, but I think networking.enableIPv6 = false is the better solution. I haven't tested it again, but I think the problems I reported previously might have been unrelated to this option (I think it was a driver problem).
Hope that helped

@SrTobi yea quite helpful thanks 🙏

regarding firewall:
the UDP port 51066 needs to be enabled - not sure yet how to configure it in the nixpkgs for the package though
-A PREROUTING -j nixos-fw-rpfilter
-A nixos-fw-rpfilter -m rpfilter --validmark -j RETURN
-A nixos-fw-rpfilter -p udp -m udp --sport 67 --dport 68 -j RETURN
-A nixos-fw-rpfilter -s <IP_ADDRESS>/32 -d <IP_ADDRESS>/32 -p udp -m udp --sport 68 --dport 67 -j RETURN

-A nixos-fw-rpfilter -p udp -m udp --dport 51066 -j RETURN

-A nixos-fw-rpfilter -j DROP

Oh alright, thanks!

OT but working alternative to packaging the non-free app:
nix-shell -p wgnord
mkdir -p /var/lib/wgnord
curl https://raw.githubusercontent.com/phirecc/wgnord/master/template.conf > /var/lib/wgnord/template.conf
wgnord l $YOUR_CREDS_SEE_GITHUB_REPO_HOW_TO_GET_IT
wgnord c germany

wgnord writes a wg-quick target to /etc/wireguard, which can be added to
networking.wg-quick.interfaces.wgnord = "/etc/wireguard/wgnord.conf"

See the GitHub repo for more information: https://github.com/phirecc/wgnord

Nord released the client in source form (GPL3) at the start of the year. I'm having partial success getting it to build (Go compiler errors), but chipping away at it. That would get us a native build.

Randomly found this on a reddit thread. Works out of the box.
https://gist.github.com/myypo/31c52196f7987ef62f54092cb07aefd7

I managed to compile it from source here if anyone is interested: https://github.com/NordSecurity/nordvpn-linux/issues/355#issuecomment-2480381945


fetch the latest changes - TypeError: 'NoneType' object is not subscriptable

          fetch the latest changes

Originally posted by @ARajgor in https://github.com/stitionai/devika/issues/47#issuecomment-2035254587
I'm having this same issue and ironically I was doing a simple tic tac to as a test.  But I'm up to date as of this-morning and this was closed 2 weeks ago I think.
Search Engine :: bing
Exception in thread Thread-6 ():
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1073, in _bootstrap_inner
self.run()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "D:\Github\devika\devika.py", line 94, in
thread = Thread(target=lambda: agent.execute(message, project_name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\agents\agent.py", line 346, in execute
search_results = self.search_queries(queries, project_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\agents\agent.py", line 106, in search_queries
link = web_search.get_first_link()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\browser\search.py", line 32, in get_first_link
return self.query_result["webPages"]["value"][0]["url"]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

Which one did u use?
Bing or duckduckgo

Can you share backend log.

Which one did u use? Bing or duckduckgo

I used bing with ollama codeqwen, its not a one time thing either happens every time the same error.
@ARajgor is this sufficient?  The is all from the cmd window I do see a log devika_agent.log is that what you are referring to or is there another log?
24.04.20 08:42:03: root: INFO   : /api/data GET
24.04.20 08:42:03: root: DEBUG  : /api/data GET - Response: {"models":{"CLAUDE":[["Claude 3 Opus","claude-3-opus-20240229"],["Claude 3 Sonnet","claude-3-sonnet-20240229"],["Claude 3 Haiku","claude-3-haiku-20240307"]],"GOOGLE":[["Gemini 1.0 Pro","gemini-pro"]],"GROQ":[["LLAMA3 8B","llama3-8b-8192"],["LLAMA3 70B","llama3-70b-8192"],["LLAMA2 70B","llama2-70b-4096"],["Mixtral","mixtral-8x7b-32768"],["GEMMA 7B","gemma-7b-it"]],"MISTRAL":[["Mistral 7b","open-mistral-7b"],["Mistral 8x7b","open-mixtral-8x7b"],["Mistral Medium","mistral-medium-latest"],["Mistral Small","mistral-small-latest"],["Mistral Large","mistral-large-latest"]],"OLLAMA":[["codeqwen","codeqwen:7b"],["cuda","cuda:latest"],["dolphincoder","dolphincoder:15b"],["llama2-uncensored","llama2-uncensored:latest"],["llama3","llama3:8b"]],"OPENAI":[["GPT-4 Turbo","gpt-4-turbo"],["GPT-3.5 Turbo","gpt-3.5-turbo-0125"]]},"projects":[],"search_engines":["Bing","Google","DuckDuckGo"]}
24.04.20 08:42:08: root: INFO   : /api/create-project POST
24.04.20 08:42:08: root: DEBUG  : /api/create-project POST - Response: {"message":"Project created"}
24.04.20 08:42:08: root: INFO   : /api/get-agent-state POST
24.04.20 08:42:08: root: DEBUG  : /api/get-agent-state POST - Response: {"state":null}
24.04.20 08:42:28: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': False, 'message': 'create simple tic tac to in python', 'timestamp': '2024-04-20 08:42:28'}}
24.04.20 08:42:28: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 738}
Model: codeqwen:7b, Enum: OLLAMA
24.04.20 08:42:53: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 837}
plan ::  Project Name: TicTacToe
Your Reply to the Human Prompter: I am going to create a simple Tic Tac Toe game in Python using command line interface.
Current Focus: Implementing the game logic and user input handling.
Plan:

[ ] Step 1: Define the board size (3x3).
[ ] Step 2: Create a function to initialize the board with empty spaces.
[ ] Step 3: Create a function to print the current state of the board.
[ ] Step 4: Implement user input validation for each move, ensuring that the player inputs a valid row and column number (1-3).
[ ] Step 5: Implement game rules to determine when a player has won or if it's a tie.
[ ] Step 6: Create a function to handle the game loop, allowing players to take turns until a winner or a tie is detected.

Summary: This plan will create a simple Tic Tac Toe game in Python using command line interface. It will include functions for initializing the board, printing the current state of the board, input validation, game rules, and handling the game loop.
24.04.20 08:42:53: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 603, 'timestamp': '2024-04-20 08:42:28'}]
24.04.20 08:42:53: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': 'I am going to create a simple Tic Tac Toe game in Python using command line interface.', 'timestamp': '2024-04-20 08:42:53'}}
24.04.20 08:42:53: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': '{\n    "1": "Define the board size (3x3).",\n    "2": "Create a function to initialize the board with empty spaces.",\n    "3": "Create a function to print the current state of the board.",\n    "4": "Implement user input validation for each move, ensuring that the player inputs a valid row and column number (1-3).",\n    "5": "Implement game rules to determine when a player has won or if it's a tie.",\n    "6": "Create a function to handle the game loop, allowing players to take turns until a winner or a tie is detected. "\n}', 'timestamp': '2024-04-20 08:42:53'}}
context_keywords ::  ['input', 'logic', 'implementing', 'game', 'handling']
24.04.20 08:42:54: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1319}
Model: codeqwen:7b, Enum: OLLAMA
24.04.20 08:42:56: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1005}
internal_monologue ::  Sure, I can start by initializing the board with empty spaces.
24.04.20 08:42:56: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 983, 'timestamp': '2024-04-20 08:42:28'}, {'internal_monologue': 'Sure, I can start by initializing the board with empty spaces.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-20 08:42:56'}]
24.04.20 08:42:56: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1630}
Model: codeqwen:7b, Enum: OLLAMA
24.04.20 08:43:00: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 889}
research ::  {'queries': ['How to implement Tic Tac Toe in Python', 'Understanding game logic for Tic Tac Toe'], 'ask_user': ''}
24.04.20 08:43:00: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': 'I am browsing the web to research the following queries: How to implement Tic Tac Toe in Python, Understanding game logic for Tic Tac Toe.\n If I need anything, I will make sure to ask you.', 'timestamp': '2024-04-20 08:43:00'}}
24.04.20 08:43:00: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 983, 'timestamp': '2024-04-20 08:42:28'}, {'internal_monologue': 'Sure, I can start by initializing the board with empty spaces.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 852, 'timestamp': '2024-04-20 08:42:56'}]
24.04.20 08:43:00: root: INFO   :
Search Engine :: bing
Exception in thread Thread-3 ():
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1073, in _bootstrap_inner
self.run()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.1008.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "D:\Github\devika\devika.py", line 94, in
thread = Thread(target=lambda: agent.execute(message, project_name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\agents\agent.py", line 346, in execute
search_results = self.search_queries(queries, project_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\agents\agent.py", line 106, in search_queries
link = web_search.get_first_link()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Github\devika\src\browser\search.py", line 32, in get_first_link
return self.query_result["webPages"]["value"][0]["url"]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

Here Bing api didn't return web result that's why it can't find the url


feat: support mutiscreen when slurp

fix: https://github.com/waycrate/wayshot/issues/8

I finally had some time to look at and test this. (Inside sway, I ran WLR_WL_OUTPUTS=2 sway to create a two-output nested instance, and then made a number of screenshots, using commands like swaymsg output WL-2 scale 1.5 and swaymsg output WL-2 transform flipped-180 to change output scales and apply output transforms.)

Hey! Thanks once again for taking a look ❤.

I've only noticed one issue so far: transformed outputs are not properly handled; the screenshot buffers need to be untransformed and composited in the right location. However, transformed outputs also having rotated screenshot images may also be an issue with the original wayshot -- or possibly even the protocol https://wayland.app/protocols/wlr-screencopy-unstable-v1 itself, which does not appear to explain whether the buffers received are transformed. Either way, I still consider this PR to be a strict improvement.

This makes me curious, does this issue occur on grim too? Would you mind testing it out? 😅 I sadly don't have access to anything apart from my laptop at the moment.

This makes me curious, does this issue occur on grim too? Would you mind testing it out?

It does not happen with grim. Under Sway, running swaymsg output '*' transform 180 to rotate the display, and capturing a screenshot with grim, produces an image which displays windows right-side-up; doing the same with Wayshot (either original, my test branch, or this PR) produces an image whose windows are upside-down.

I think we need to get the rotation of the screen, before make them one image, it need to rotate the image to the right place

I try to solve transform in my repo,
https://github.com/Decodetalkers/haruhishot/commit/8bc7f4d214026a44a4b27fa323e92981e6f27017
but there is still some problem , when rotate with 90 and 270, it cannot shot the right place, I need to see how grim handle it

it seems something wrong with capture_region, I think it is the bug of sway, because rotate 90 degree, capture region from 0, 0, 100, 100, it captures from 2400, 1000, -100, -100, and grim does not use capture_region. it capture all the screens, and caculate where to cut, finally makes an image

I see I'll try and test this theory.

introduction is here, but I do not know what is a good way to save them in OutputInfo, because they are all in wl_output. so ..

I think we can query the transform and keep it in a transform field right?

Completed in: https://github.com/waycrate/wayshot/pull/41

Thank you for all your help and contributions ❤


fix: deployment failing due to /etc/containers/systemd dir missing

When we upgrade the module from 0.6.7 to 0.7.1 our deployment start to fails because the new code try to create
/etc/containers/systemd/users while /etc/containers/systemd doesn't exists.
This commit fix the issue be ensuring that /etc/containers/systemd also exists.

Thanks for the feedback


Update the projection head (normalization and size).

TL;DR: current projection head doesn't do what it is supposed to do. It should have a batch norm and the size of the features and projections may be reduced further.
In the previous implementations of contrastive learning models (dynacontrast), we used batch norm in the projection head after each MLP.
(projection): Sequential(
    (fc1): Linear(in_features=2048, out_features=2048, bias=False)
    (bn1): BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu1): ReLU()
    (fc2): Linear(in_features=2048, out_features=128, bias=False)
    (bn2): BatchNorm1dNoBias(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )

This paper also recommends using a non-linear projection head with batch norm. Different projection heads are evaluated by comparing the rank of the features (# of independent features) before and after the projection head:

As expected, the rank(projections) << rank(features).
Our current model's behavior is the opposite: rank(projections)> rank(features) as seen from the examination of the principal components in each.

This seems to be the consequence of clipping of projections, which seems to be due to the use of ReLU without normalization.
plt.plot(np.mean(embedding_dataset["projections"].values,axis=1))

plt.plot(np.std(embedding_dataset["projections"].values,axis=1))

@mattersoflight In SimCLR and others using InfoNCE-style losses, there is an implicit L2-normalization of $\mathbb{z}$ happening in the loss function, since they use cosine similarity as the distance function. The triplet margin loss uses L2 distance (which is fully determined by the cosine and vice versa for unit vectors). I still need to find a reference implementation, but the planned removal of L2 normalization might not be needed.

@ziw-liu  L2 normalization is indeed equivalent to converting feature vectors into unit vectors. It also makes sense that the loss is computed between unit vectors (either cosine similarity or eucledian) given SimCLR paper.
I agree that L2 normalization of projections doesn't need to be removed. NO NEED to implement that as an argument.
[TripletMarginLoss](https://pytorch.org/docs/stable/generated/torch.nn.TripletMarginLoss.html) provided by torch.nn doesn't do normalization by default.

@ziw-liu the paper that compares different types of projection heads uses infoNCE loss (which reads the same as NT-Xent loss in simCLR). It may be that using NT-Xent loss promotes higher-rank embeddings.

Model changes was implemented in #145. But #154 can potentially fix the low-rank feature map.


Fix(searchbar) prevent keyboard from closing

(click) and (mousedown) events are both fired when pressing clear(X) button, this causes a range of issues including not allowing the keyboard to close after clear has been pressed. This is due to the fact that _shouldBlur is left in the false state after clear rather then being reset to true. Issue is as follows.
Short description of what this resolves:
This issue

Clear is pressed

clearInput from (mousdown) fires _shouldBlur = false

inputBlurred fires from deselecting input _shouldBlur = true

clearInput from (click) fires _shouldBlur = false

Next blur will now refocus input do to shouldBlur = false

Changes proposed in this pull request:

Ionic Version: 1.x / 2.x / 3.x / 4.x
Fixes: #

Hi there! We're going to be closing this as the repo is now organized for V4. For V3 related issues/PRs, please direct them here
https://github.com/ionic-team/ionic-v3
Thanks!


Create dmacc.txt

@demiwa000 Please rename lib/domains/edu/lib/domains/edu/dmacc.txt to lib/domains/edu/dmacc.txt

@philipto I believe I fixed it? I'm new to both Github and Java so this is all a bit confusing.

@demiwa000 Yes, it is now fine. Pull request merged. Please notice that free educational license for JetBrains tools is for high and higher schools students, i.e. those who are over 13 of age.
You can start requesting your free license at https://www.jetbrains.com/shop/eform/students in about 30 minutes, please wait while the changes propagate through our system.
Thank you!


Reintroduce Runtime Identifier in Pubxml to Resolve Platform Mismatch

This PR resolves the NETSDK1032 error ("The RuntimeIdentifier platform 'win-x64' and the PlatformTarget 'x86' must be compatible") encountered when building and publishing WinUI 3 applications with multiple target architectures.
Context:
The error occurred on x64 and arm64 machines when selecting both x86 and x64 packages or when building C# WinUI 3 apps with .NET 9.0 set to x64 or x86 platforms.
Fix:
To resolve the issue, the RuntimeIdentifier (RID) has been reintroduced in the pubxml file, explicitly setting it to win-{arch} to ensure compatibility between the target platform and architecture. Conditional RIDs have also been removed across all project templates in favour of non-portable RIDs.

/azp run


Crash in internalInfoCallback

I've got this crash more than a couple of times in an app, Shifty, using AXSwift
Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       EXC_I386_GPFLT
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [576]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libswiftCore.dylib            	0x00000001093e14e1 swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::incrementSlow(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 33
1   libswiftCore.dylib            	0x00000001093c9984 _swift_retain_(swift::HeapObject*) + 68
2   org.cocoapods.AXSwift         	0x0000000108f04078 specialized internalInfoCallback(_:axElement:notification:cfInfo:userData:) + 552
3   org.cocoapods.AXSwift         	0x0000000108f034ee @objc internalInfoCallback(_:axElement:notification:cfInfo:userData:) + 78
4   com.apple.HIServices          	0x00007fff4b190672 _XXMIGPostNotification + 821
5   com.apple.HIServices          	0x00007fff4b19ddac _XPostNotification + 351
6   com.apple.HIServices          	0x00007fff4b172abe mshMIGPerform + 212
7   com.apple.CoreFoundation      	0x00007fff4c9cc6a9 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 41
8   com.apple.CoreFoundation      	0x00007fff4c9cc607 __CFRunLoopDoSource1 + 527
9   com.apple.CoreFoundation      	0x00007fff4c9b4689 __CFRunLoopRun + 2574
10  com.apple.CoreFoundation      	0x00007fff4c9b3a28 CFRunLoopRunSpecific + 463
11  com.apple.HIToolbox           	0x00007fff4bc4cb35 RunCurrentEventLoopInMode + 293
12  com.apple.HIToolbox           	0x00007fff4bc4c774 ReceiveNextEventCommon + 371
13  com.apple.HIToolbox           	0x00007fff4bc4c5e8 _BlockUntilNextEventMatchingListInModeWithFilter + 64
14  com.apple.AppKit              	0x00007fff49f08eb7 _DPSNextEvent + 997
15  com.apple.AppKit              	0x00007fff49f07c56 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1362
16  com.apple.AppKit              	0x00007fff49f01cb9 -[NSApplication run] + 699
17  com.apple.AppKit              	0x00007fff49ef13f7 NSApplicationMain + 780
18  io.natethompson.Shifty        	0x0000000108e022a9 0x108e00000 + 8873
19  libdyld.dylib                 	0x00007fff79b9608d start + 1

It doesn't always has the same backtrace, but internalInfoCallback seems to be the cause for all of them.
Looking at the disassembly the crash seems to be caused by the line
    observer.callbackWithInfo!(observer, element, notif, info)

The observer, or the callback aren't valid I think.
This is probably because I'm not using it correctly in the app Shifty (I would actually ask you what is the correct way to stop an observer), but we should probably not crash either way, and probably check if the observer or the callback are valid?
The offending code in Shifty is in this file https://github.com/thompsonate/Shifty/blob/5d58e021827b77fb00b850eb522fa6b73c158626/Shifty/BrowserManager.swift in the stopBrowserWatcher. I wonder if I should remove the notifications instead of calling stop.

Hmm, we should have stopped watching on Observer deinit. Merging a fix soon. Thanks for your report!

I'm still getting this issue in Shifty with AXSwift 0.2.3.
Looks like the offending line is still Observer.swift:162: observer.callbackWithInfo!(observer, element, notif, info) in Observer.swift. Not sure if it's the root cause, but is there any reason to be force unwrapping there?
Here's BrowserManager.swift:171 referenced in my crash report:
Here's the stack trace from my crash reporter:
Shifty                closure #1 (AXSwift.Observer, AXSwift.UIElement, AXSwift.AXNotification, [Swift.String : Swift.AnyObject]?) -> () in static Shifty.BrowserManager.(startBrowserWatcher in _BCA98A1A69929549465A957C57870AC7)(_: Swift.Int32, callback: () -> ()) throws -> () BrowserManager.swift:171
Shifty                partial apply forwarder for closure #1 (AXSwift.Observer, AXSwift.UIElement, AXSwift.AXNotification, [Swift.String : Swift.AnyObject]?) -> () in static Shifty.BrowserManager.(startBrowserWatcher in _BCA98A1A69929549465A957C57870AC7)(_: Swift.Int32, callback: () -> ()) throws -> () <compiler-generated>:0
AXSwift               function signature specialization <Arg[0] = Dead> of AXSwift.(internalInfoCallback in _1C4E6C7EFC73FEB542EA8C20A2603D9E)(_: __C.AXObserverRef, axElement: __C.AXUIElementRef, notification: __C.CFStringRef, cfInfo: __C.CFDictionaryRef, userData: Swift.UnsafeMutableRawPointer?) -> () Observer.swift:162
AXSwift               @objc AXSwift.(internalInfoCallback in _1C4E6C7EFC73FEB542EA8C20A2603D9E)(_: __C.AXObserverRef, axElement: __C.AXUIElementRef, notification: __C.CFStringRef, cfInfo: __C.CFDictionaryRef, userData: Swift.UnsafeMutableRawPointer?) -> () <compiler-generated>:0
HIServices            0x7fff2cbe7000 + 145281
HIServices            0x7fff2cbe7000 + 200325
HIServices            0x7fff2cbe7000 + 23777
CoreFoundation        0x7fff2e34b000 + 338679
CoreFoundation        0x7fff2e34b000 + 338517
CoreFoundation        0x7fff2e34b000 + 240188
CoreFoundation        0x7fff2e34b000 + 237070
HIToolbox             0x7fff2d667000 + 43483
HIToolbox             0x7fff2d667000 + 42773
HIToolbox             0x7fff2d667000 + 42150
AppKit                0x7fff2b9f1000 + 110587
AppKit                0x7fff2b9f1000 + 105875
AppKit                0x7fff2b9f1000 + 81584
AppKit                0x7fff2b9f1000 + 13296
Shifty                main AppDelegate.swift:21
libdyld.dylib         0x7fff5a7cc000 + 91093

The issue isn't Observer.swift:162; from the backtrace, it's within your handler. Possibly one of the force casts or unwraps in attribute() is failing. These aren't supposed to fail (obviously).
In UIElement.swift in AXSwift, try replacing this line:
https://github.com/tmandry/AXSwift/blob/1e2fa1fe911e1d4a873b580d8ee26731067000d8/Sources/UIElement.swift#L181
with:
        guard let unwrapped = value else {
            fatalError("attribute \(attribute) of \(self) was nil")
        }

        let unpacked = unpackAXValue(unwrapped)
        guard let casted = unpacked as? T else {
            fatalError("attribute \(attribute) of \(self) was unexpected type: \(String(describing: value))")
        }
        return casted

That way if you hit it again, you'll get more useful errors. Or you'll rule out that as the source of the crash.
To answer your question, the reason the code force-unwraps in Observer.swift is because if we are receiving events, the observer must still be valid. If it's not, checking one of its fields isn't going to help. Checking whether a field on it has gone nil that should NEVER go nil means we just noticed we were reading invalid memory, and quietly ignored it, instead of failing loudly which we should be doing.

If you do get anything from the code I pasted above, please open a new issue and report all the output you get. That would definitely be a bug in AXSwift and we should be handling these cases more gracefully if they do happen.
Re-closing for now because I think the original issue here is fixed.


Improve ln.setup.settings docstring

https://docs.lamin.ai/lamindb.setup.settings

This doesn't look great.

Ok, this happens by default because it shows __repr__ of settings as runtime.
@falexwolf do we have a way to understand that a doc building session is running at runtime? Then we can correct here at runtime to avoid this. Maybe an env variable or something like this to check if the docs are being built now?

A bit hacky but I'm using this in bionty: https://github.com/laminlabs/bionty/blob/main/bionty/models.py#L413

Thanks, i think checking for sphynx is fine here also.


Reactivate delete artifact

The action GeekyEggo/delete-artifact  has been released in v5.
This fixes the need of a token and we can use it again for PRs.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 78.435%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
451

Relevant Lines:
575

💛  - Coveralls

Is the version of delete-artifact raised to v5?

Is the version of delete-artifact raised to v5?

Yes, released yesterday


add coveralls

We'll see how this works.

Current coverage is 92.85% (diff: 100%)

No coverage report found for master at 9b1653b.

Powered by Codecov. Last update 9b1653b...c22e5e8


Issue with Crafter and Block ID's

Using Version 5.03 for MC 1.10, along with ExtraUtils2 v1.0 for MC1.10
When using Crafter Tier II or III, with multiple recipes the crafter was "eating" blocks.  Specifically I was using it to in conjunction with ExtraUtilities2 to recursively compress cobble stone blocks.  I realized the single, double, triple, etc compressed cobble uses same block ID with "meta data" distinguishing the level of compression; e.g. #4370/0, 4370/1, 4370/2 ....  So the crafter used triple compressed in its inventory to make double compressed and so on, thinking it was 1-to-1 equivalent of single compressed cobble since they have same block id number.  It would be really really great if the crafter could be updated to use text based block ID's to avoid this issue or if we could toggle ignoring/considering the extra meta-data.

Hmm the crafter should look at block ID's. I have to check this. But can you confirm this also works with latest rftools + mcjtylib as some changes to inventory handling have been done

Ok, I found it. Will be fixed next release


adding logic to auto aprv submission without cmnt

Issue #:
https://github.com/bcgov/met-public/issues/1370
Description of changes:
changes to auto approve submissions which:

do not have a textarea or textfield in the submission json
has textarea or textfield in the submission json but have a null value

changes to exclude auto approved submissions from comment listing page as they do not have a comment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the met-public license (Apache 2.0).

Codecov Report

Merging #1388 (8599230) into main (2a8e353) will decrease coverage by 0.08%.
The diff coverage is 43.47%.

@@            Coverage Diff             @@
##             main    #1388      +/-   ##
==========================================
- Coverage   71.57%   71.50%   -0.08%
==========================================
  Files         317      317
  Lines        9134     9156      +22
  Branches      644      644
==========================================
+ Hits         6538     6547       +9
- Misses       2483     2496      +13
  Partials      113      113

Flag
Coverage Δ

metapi
79.08% <43.47%> (-0.20%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

met-api/src/met_api/models/submission.py
62.28% <43.47%> (-5.12%)
:arrow_down: