Ionic cordova resources Does not generate Windows 10 UWP resources

From @tryadelion on May 25, 2017 14:13
Ionic version:  (check one with "x")
[ ] 1.x (For Ionic 1.x issues, please use https://github.com/driftyco/ionic-v1)
[ X] 2.x
[X ] 3.x
I'm submitting a ...  (check one with "x")
[ X] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Running "ionic cordova resources" does  not generate the desired resources, it says "no platforms installed", even when the platform is correctly listed on "ionic cordova platform ls"
Copied from original issue: driftyco/ionic#11785

Duplicate. Also not a bug...
https://github.com/driftyco/ionic-cli/issues/2174

@dwieeb  Windows is a platform supported by the ionic 2 RELEASE, and thus, it should be working accordingly. As i stated, it doesn't work on any of the CLI's, be it 2.0 or 3.0.
Here is the size of what the generator should generate, cutting around the central XY axis, on every SCALE-100 setting:
ICONS
24x24
44x44
50x50
71x71
150x150
310x150
310x310
SPLASH
620x300
there's also each size scaled to 125%, 150%, 200%, and 400%, but those are optional.

The framework has full support for Windows. The completely free image resizing service we offer in the CLI has a different support policy. You can track this feature request in https://github.com/driftyco/ionic-cli/issues/2174.


Error preparing for challenge answer

I have two machines which I have been using to run Win ACME. I recently upgraded to 2.1.4 with the Azure plugin. On my Windows 10 machine, I have no issue. I have to create all my certificates manually which can be a headache for a server that has a bunch of URLs like Skype. However, on my IIS Server running Windows Server 2012, I get the error in the subject line. I initially tried running using IIS to pull in the URL but that failed so I went back to doing it manually and that also failed. As far as I can tell win acme running on Windows Server 2012 trying to use Azure plugin doesn't work.
A couple of things I tried was first to copy the working installation from my windows 10 computer. Next, I tried to create the TXT files manually and that did work but defeats the purpose of using the Azure Plugin.
I am hoping to automate my Skype and Exchange server within the next couple of weeks so any help would be appreciated.

Hi @GibbyOH, please do a run with the --verbose switch on and post the log output here so that we can see what's leading up to this error.

Maybe same issue as #1414

Looks like it is trying to find a JSON file:
[VERB] Request completed with status OK
[INFO] Authorize identifier: dialin.gibbyoh.com
[INFO] Authorizing dialin.gibbyoh.com using dns-01 validation (Azure)
[EROR] Error preparing for challenge answer
Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: P
arameters: Connection String: [No connection string specified], Resource: https:
//management.azure.com/, Authority: . Exception Message: Tried the following 3 m
ethods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https
://management.azure.com/, Authority: . Exception Message: Tried to get token usi
ng Managed Service Identity. Unable to connect to the Managed Service Identity (
MSI) endpoint. Please check that you are running on an Azure resource that has M
SI setup.
Parameters: Connection String: [No connection string specified], Resource: https
://management.azure.com/, Authority: . Exception Message: Tried to get token usi
ng Visual Studio. Access token could not be acquired. Visual Studio Token provid
er file not found at "C:\Users\josh.admin\AppData\Local.IdentityService\AzureSe
rviceAuth\tokenprovider.json"
Parameters: Connection String: [No connection string specified], Resource: https
://management.azure.com/, Authority: . Exception Message: Tried to get token usi
ng Azure CLI. Access token could not be acquired. 'az' is not recognized as an i
nternal or external command,
operable program or batch file.
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAu
thResultAsyncImpl(String resource, String authority, CancellationToken cancellat
ionToken)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAu
thenticationResultAsync(String resource, String tenantId, CancellationToken canc
ellationToken)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAc
cessTokenAsync(String resource, String tenantId, CancellationToken cancellationT
oken)
at PKISharp.WACS.Plugins.ValidationPlugins.Dns.Azure.GetClient()
at PKISharp.WACS.Plugins.ValidationPlugins.Dns.Azure.CreateRecord(String reco
rdName, String token)
at PKISharp.WACS.Plugins.ValidationPlugins.DnsValidation`1.PrepareChallenge()
at PKISharp.WACS.Plugins.ValidationPlugins.Validation`1.PrepareChallenge(ICha
llengeValidationDetails challenge)
at PKISharp.WACS.RenewalExecutor.Authorize(ILifetimeScope execute, RunLevel r
unLevel, ValidationPluginOptions options, TargetPart targetPart, Authorization a
uthorization)

Definitely looks like an authentication problem, but that message about the .json file is only the result of one of three methods that it (unsuccesfully) tried.
How are you authenticating? Using a service principal account or a managed resource identity? It kind of looks like you're trying to use a managed resource identity but it's not set up properly.

I am using a managed resource identity the same as on my Windows 10 machine.

I'm afraid my knowledge of how this method of authentication works under the hood and why it can fail is limited, maybe someone with more Azure experience could chime in?

Alright, so I figured it out. On my Windows 10 and Windows Server 2016 boxes, I can just enter the Display name (ie letsencrypt), anything older I have to enter the clientID string.

Thanks for sharing that, would definitely be worth documenting!

Added a note about this issue to the docs, thanks again.


Postgres connection

I'm trying to link the superset to postgres, but when I try to import a dashboard it crashes. I also can't save charts.
Screenshots

How to reproduce the bug

Go to Import Dashboard
Click on Upload
See error.

Environment

superset version: 1.0.1
python version: Python 3.8.5
postgres version: psql (PostgreSQL) 12.6 (Ubuntu 12.6-0ubuntu<IP_ADDRESS>)
Instaled from Scratch

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[ x] I have reproduced the issue with at least the latest released version of superset.
[ x] I have checked the issue tracker for the same issue and I haven't found one similar.

Log
ERROR:superset.app:Exception on /superset/import_dashboards [POST]
Traceback (most recent call last):
File "/root/superset/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/root/superset/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/superset/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/superset/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/root/superset/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/root/superset/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/root/superset/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
File "/root/superset/lib/python3.8/site-packages/superset/utils/log.py", line 164, in wrapper
value = f(*args, **kwargs)
File "/root/superset/lib/python3.8/site-packages/superset/views/core.py", line 660, in import_dashboards
databases = db.session.query(Database).all()
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3535, in iter
return self._execute_and_instances(context)
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3556, in _execute_and_instances
conn = self._get_bind_args(
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3571, in _get_bind_args
return fn(
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3550, in _connection_from_session
conn = self.session.connection(**kw)
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1142, in connection
return self._connection_for_bind(
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1150, in _connection_for_bind
return self.transaction._connection_for_bind(
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 409, in _connection_for_bind
self._assert_active()
File "/root/superset/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 289, in _assert_active
raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: 'ascii' codec can't encode character '\xe1' in position 7: ordinal not in range(128) (Background on this error at: http://sqlalche.me/e/13/7s2a)
INFO:werkzeug:<IP_ADDRESS> - - [17/Feb/2021 18:50:15] "POST /superset/import_dashboards HTTP/1.1" 500 -
INFO:werkzeug:<IP_ADDRESS> - - [17/Feb/2021 18:54:59] "GET /superset/import_dashboards HTTP/1.1" 200 -

@betodealmeida you're the most familiar with import - thoughts?

@betodealmeida If you have the most familiarity with the import function - please help. We are trying to export dashboard from Ver 0.34 to 1.0 and facing the same "unknown error" issue.


Update readme from branch example

Updates
I updated the readme file. Please review and accept.

Thought
I probably shouldn't, but okay. 👍


Mapping interface hierarchy

Hi there,
im having difficulties to use ModelMapper to map our JPA-Entity tree to our TO object tree. We are working with TO-Java-Interfaces and are hiding implementations of TOs to our service-class. This class maps the entities by using self-written converters classes. These converters are instantiating correct implementation of TO-Interface and are filling the data. The converters are using other converters to map the whole object hierarchy.
W'd like to exchange our converters with ModelMapper and I tried to use ModelMapper within one of our services:
Here is my interface TO setup:
ISampleTO has ISub1TO and ISub2TO. ISub1TO has ISub3TO.
interface ISampleTO {
	Long getId();
	void setId(Long id);

	ISub1TO getSub1TO();
	void setSub1TO(ISub1TO id);

	ISub2TO getSub2TO();
	void setSub2TO(ISub2TO id);
	...
}

interface ISub1TO {
	Long getId();
	void setId(Long id);

	ISub3TO getSub3TO();
	void setSub3TO(ISub3TO id);
	...
}

interface ISub2TO {
	Long getId();
	void setId(Long id);
	...
}

interface ISub3TO {
	Long getId();
	void setId(Long id);
	...
}

same hierarchy with SampleEntity, but with Entity instead of TO in name.
Service class has private Methods to encapsulate ModelMapper:
private SampleEntity createFromDto(ISampleTO to) {
    return modelMapper.map(to, SampleEntity.class);
}

private ISampleTO createFromEntity(SampleEntity entity) {

    ISampleTO newTo = new SampleTO();
    modelMapper.map(entity, newTo);
    return newTo;
}

When I try to run this I'm getting the following error from ModelMapper:
Failed to instantiate instance of destination ISub1TO. Ensure that ISub1TO has a non-private no-argument constructor.

I found issue https://github.com/modelmapper/modelmapper/issues/13 but as far as I understand ModelMapper does not support mapping of nested interface-TOs. Is this correct? The last post refers to "TypeMap inheritance", but when I read the documentation I can not see any helpful information to a solution of my problem.
My questions are:

Can ModelMapper map a hierarchy of interfaces?
If yes, how can I tell ModelMapper which realization it should be used to map to (interfaces can have more than one implementation)
If yes, how would this be achieved, I don't have a clue.

Thank you very much.

Hi @Divisibility ,
We have Type Map Inheritance that was used for solving this kind of problems.
I guess your entity will looks like below.
class SampleEntity {
  Long id;
  Sub1 sub1;
  Sub2 sub2;
  Sub3 sub3;
}

And you need to configure your ModelMapper as below.
modelMapper.typeMap(Sub1.class, Sub1TO.class).include(ISub1TO.class);
modelMapper.typeMap(Sub2.class, Sub2TO.class).include(ISub2TO.class);
modelMapper.typeMap(Sub3.class, Sub3TO.class).include(ISub3TO.class);

Hi @chhsiao90,
I have tested your suggestion and unfortantely it does not work. Im getting the same errors:

Failed to instantiate instance of destination me.dames.to.ISub1TO. Ensure that me.dames.to.ISub1TO has a non-private no-argument constructor.
Caused by: java.lang.NoSuchMethodException: me.dames.to.ISub1TO.()
..
Failed to instantiate instance of destination me.dames.to.ISub1TO. Ensure that me.dames.to.ISub1TO has a non-private no-argument constructor.
Caused by: java.lang.NoSuchMethodException: me.dames.to.ISub1TO.()
..
Failed to instantiate instance of destination me.dames.to.ISub2TO. Ensure that me.dames.to.ISub2TO has a non-private no-argument constructor.
Caused by: java.lang.NoSuchMethodException: me.dames.to.ISub2TO.()
..

I have attached my sample project as zip and I'd appreciate some more assistence a lot!
Thank you.


Telescope-Nova - Positional projection does not match the query document

In Nova, can't we not have an array of objects in the schema? something like..
/**
 * castElements schema
 * @type {SimpleSchema}
 */
Telescope.schemas.castElements = new SimpleSchema({
  celeb: {
    type: String,
    optional: true,
    publish: true,
  },
  castedAs: {
    type: String,
    optional: true,
    publish: true,
  }
});

...
  cast: {
    type: [Telescope.schemas.castElements],
    publish: true,
    optional: true,
  },

I am getting the exception posts.single :  Positional projection 'cast.$.celeb' does not match the query document. on subscription.
am i missing something or i need to change my schema?

This is a MongoDB error, so I don't think it has anything to do with your schema. It's that your update call isn't formatted correctly.


Sync Fork

[!IMPORTANT]
Please ensure that you are basing this PR on the OctopusDeploy/sumologic-azure-function octopus branch and not master from the source Sumo repo.

Background

Results
Technical details

How to review this PR

Quality :heavy_check_mark:

Pre-requisites

[ ] I have referenced this pre-release build of sumo-azure-function in a CloudPortal branch for E2E tests (All 3 Azure Function Package URLs)
[ ] I have referenced this pre-release build of sumo-azure-function in a Dynamic Workers branch for E2E tests (All 3 Azure Function Package URLs)
[ ] I understand the process for rolling out new Sumo Azure Functions for NSG Flow Logs
[ ] Are you adding a new dependency?

[ ] Ensure the dependency will be detected by Renovate or document it as an exception in Dependency Management

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 3 committers have signed the CLA.:white_check_mark: bradleyjford:x: priyansh-sumo:x: himanshu219You have signed the CLA already but the status is still pending? Let us recheck it.


system variable (user only) OLLAMA_MODELS is ignored Worked in 0.2.8

What is the issue?
I had custom location of ollama model in 2.8 but after 3.0 update it is ignored.
 ollama run gemma2:27b
pulling manifest
pulling d7e4b00a7d7a...   1% ▕                ▏ 157 MB/ 15 GB  768 KB/s   5h35m
Error: max retries exceeded: write C:\Users\danda\.ollama\models\blobs\sha256-d7e4b00a7d7a8d03d4eed9b0f3f61a427e9f0fc5dea6aeb414e41dee23dc8ecc-partial: Na disku není dost místa.

OS
Windows
GPU
Nvidia
CPU
Intel
Ollama version
0.3.0

What did you do to update?  Did you restart ollama?

I updated 2 hours ago or so. I did not restart ollama after updating to 0.3.0 since it was restarted by installer. I restarted it 2 days ago when I set up ollama so the variable took effect (it did). No restarts or variable changes since then.
From: frob @.>
Sent: Friday, July 26, 2024 12:02 PM
To: ollama/ollama @.>
Cc: mcDandy @.>; Author @.>
Subject: Re: [ollama/ollama] system variable (user only) OLLAMA_MODELS is ignored Worked in 0.2.8 (Issue #5978)
What did you do to update? Did you restart ollama?
—
Reply to this email directly, view it on GitHub https://github.com/ollama/ollama/issues/5978#issuecomment-2252412901 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AEN2KD4Q4IUC7R3RHHNTX73ZOINCVAVCNFSM6AAAAABLQEFZ7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJSGQYTEOJQGE .
You are receiving this because you authored the thread.  https://github.com/notifications/beacon/AEN2KD524SCG3U6TRL4SNY3ZOINCVA5CNFSM6AAAAABLQEFZ7SWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUGIEL6K.gif Message ID: @.*** @.***> >

Sorry I did not understand the question. I just ran the installer since variable was setup. No additional restarts done by me.

Server logs will show what OLLAMA_MODELS is set to, if you add some logs we can check.  If it's set to F:\Users\danda but the server is writing to C:\Users\danda then that is indeed a problem.  I have tested it locally, ollama 0.3.0 on a linux system does respect the setting of OLLAMA_MODELS.

I am on windows...
server.zip

2024/07/23 21:50:01.262+02:00 level=INFO source=routes.go:1147 msg="Listening on <IP_ADDRESS>:11434 (version 0.2.8)"
2024/07/23 21:50:01 routes.go:1100: INFO server config env="map[ OLLAMA_MODELS:C:\\Users\\danda\\.ollama\\models OLLAMA_NOHISTORY:false
2024/07/23 23:09:37.655+02:00 level=INFO source=routes.go:1147 msg="Listening on <IP_ADDRESS>:11434 (version 0.2.8)"
2024/07/23 23:09:37 routes.go:1100: INFO server config env="map[ OLLAMA_MODELS:F:\\Users\\danda\\OneDrive - Univerzita Pardubice\\Dokumenty\\LLMs OLLAMA_NOHISTORY:false
2024/07/26 11:03:09.968+02:00 level=INFO source=routes.go:1146 msg="Listening on <IP_ADDRESS>:11434 (version 0.3.0)"
2024/07/26 11:03:09 routes.go:1099: INFO server config env="map[ OLLAMA_MODELS:C:\\Users\\danda\\.ollama\\models OLLAMA_NOHISTORY:false
2024/07/26 11:40:19.347+02:00 level=INFO source=routes.go:1146 msg="Listening on <IP_ADDRESS>:11434 (version 0.3.0)"
2024/07/26 11:40:19 routes.go:1099: INFO server config env="map[ OLLAMA_MODELS:F:\\Users\\danda\\OneDrive - Univerzita Pardubice\\Dokumenty\\LLMs OLLAMA_NOHISTORY:false

The first time you restarted 0.3.0 at 2024/07/26 11:03:09, OLLAMA_MODELS was set to C:.  The second time at 2024/07/26 11:40:19 it is set to F:.  If you haven't restarted since, ollama pull should download the model to F:.  Since it alternates, I am guessing you have two different ways of updating/restarting ollama.  Which ever way it is, the one you did at 2024/07/26 11:40:19 is the better way.

I am allways updating through the installer and allways starting with ollama run gemma2:27b. Wierd thing is that I did nothing different from the beginning and everything was exactly the same. I just restarted PC before the last run.

Well apart of the very first run where I did not have the variable set.

@mcDandy the simplest way to make sure things are totally set up would be a reboot, but that's not strictly necessary as long as you restart your shells and the ollama tray application.  What is probably going on is you set the environment variable, but still had a cmd or powershell terminal running with the old environment, and at some point you quit the Ollama tray application, and ran the CLI in one of those "stale" terminals, which caused Ollama to restart with the old environment.


Using with Karma + Webpack

I'm having a hard time running this plugin as part of my Karma test run. I'm using webpack as a preprocessor. I figured the types would be stripped by Webpack's Babel loader, but that is not happening. This is the error I'm getting when I run the tests:
Module build failed: TypeError: ../autocomplete/index.js: Cannot read property 'replace' of null
  at Buffer.push (../node_modules/babel-core/lib/generation/buffer.js:273:16)

I tried both with and without the plugin configured as a loader in my webpack configuration file:
loaders: ['babel?plugins[]=rewire,plugins[]=tcomb']

But that doesn't seem work. I'm at a loss.

Hi @volkanunsal

I tried both with and without the plugin configured as a loader in my webpack configuration file

Not sure I'm follow: your karma configuration throws an error without babel-plugin-tcomb?
Anyway this is a configuration I'm using in a project of mine and works fine if can be useful:
devDependecies
{
  "babel": "5.8.34",
  "babel-core": "5.8.34",
  "babel-loader": "5.3.2",
  "babel-plugin-tcomb": "0.1.3",
  "isparta-loader": "1.0.0",
  "karma": "0.13.9",
  "karma-chrome-launcher": "0.2.0",
  "karma-coverage": "0.5.3",
  "karma-firefox-launcher": "0.1.4",
  "karma-tap": "1.0.3",
  "karma-webpack": "1.7.0",
  "tape": "4.2.2",
  "webpack": "1.12.9"
}

karma.conf.js
var webpack = require('webpack')
var path = require('path')

module.exports = function getConfig(config) {
  config.set({
    browserNoActivityTimeout: 30000,
    browsers: [process.env.CONTINUOUS_INTEGRATION ? 'Firefox' : 'Chrome'],
    singleRun: true,
    frameworks: ['tap'],
    files: [
      'test/index.js'
    ],
    preprocessors: {
      'test/index.js': ['webpack']
    },
    webpack: {
      node: {
        fs: 'empty'
      },
      module: {
        preLoaders: [
          // babel
          {
            test: /\.js$/,
            exclude: [
              path.resolve('src/'),
              path.resolve('node_modules/')
            ],
            loader: 'babel',
            query: { plugins: ['tcomb'] }
          },
          // code coverage
          {
            test: /\.js$/,
            include: path.resolve('src/'),
            loader: 'isparta'
          }
        ]
      },
      plugins: [
        new webpack.DefinePlugin({
          'process.env.NODE_ENV': JSON.stringify('test')
        })
      ]
    },
    webpackMiddleware: {
      noInfo: true
    },
    reporters: ['dots', 'coverage'],
    coverageReporter: {
      type: 'text'
    }
  })
}

I narrowed down the error to the fact that some type was being used for a function declared inside another function. When I moved the function declaration to the top level, it magically started working. I'm not sure if this is a Babel issue, but it sounds like it might be.


make package name "default" reserved

@rabbah - fail any new packages created with the name default. regarding existing packages having the name default - leave them be for now?


Eliminate Carbon

This PR finishes removing the Carbon component system, which was a hasty original choice. #90
In the short term, this gives us much easier full control over everything. Longer term, if we need to adopt the govuk frontend styles, this frees us up to do it.
Some style changes visible below. Very happy to make any changes.
https://user-images.githubusercontent.com/1664407/232755298-db7dddf3-3138-49eb-9aaa-ebcb4ba75e7c.mp4

Amazing work Dustin!


No output from shap.force_plot()

I am not getting any output from force_plot. Here is my code:
explainer = shap.TreeExplainer(cboost)
shap_values = explainer.shap_values(X_test)
shap.initjs()
shap.force_plot(explainer, shap_values, X_test)

Please provide the shap version and the dataset so we could reproduce your issue. Thank you.

check shap values, chances are all of them are 0. u will not get any plot then.


Fails to open browser if running through torsocks

Steps to reproduce:

torsocks cabal repl
import Web.Browser
openBrowser "http://haskell.org/" >>= print

The above only prints True, but nothing opens. The above works (Firefox opens haskell.org if not running through torsocks.

I just noticed that torsocks xdg-open "http://haskell.org" doesn't open on its own. So I guess this isn't necessarily a problem with your code, at all.


Fix simclr loss to work with distributed training.

fix  #258.
Also, while fixing this I noticed the current implementation is wrong even on a single gpu:
The paper states:
"The final loss is computed across all positive pairs, both (i, j) and (j, i)" meaning the loss is computed twice, once (za,zb) and once (zb,za)
To check that this works I tried to run this code on a multi gpu (4) instance:
from tensorflow_similarity.losses.simclr import SimCLRLoss
import tensorflow as tf

global_batch_size=512

h1 = tf.random.normal((global_batch_size,256))
h2 = tf.random.normal((global_batch_size,256))

l = SimCLRLoss()(h1,h2)
assert l == tf.reduce_sum(SimCLRLoss(reduction=tf.keras.losses.Reduction.NONE)(h1,h2))/global_batch_size

dataset = tf.data.Dataset.from_tensor_slices((h1,h2)).batch(global_batch_size)
bh1,bh2 = next(iter(dataset))
assert l == SimCLRLoss()(bh1,bh2)

strategy = tf.distribute.MirroredStrategy(devices=["GPU:0", "GPU:1","GPU:2", "GPU:3"])
dist_dataset = strategy.experimental_distribute_dataset(dataset)

def replica_fn(input):
    h1,h2 = input
    return SimCLRLoss(reduction=tf.keras.losses.Reduction.NONE)(h1,h2)
x = next(iter(dist_dataset))
result = strategy.run(replica_fn, args=(x,))

l_distributed = tf.reduce_sum(strategy.reduce("SUM", result, axis=None))/global_batch_size
abs(l-l_distributed)

>>> <tf.Tensor: shape=(), dtype=float32, numpy=9.536743e-07>

As you see the result is not bit exact (e-7) but this is the exact result compared to original implementation.
Im not sure how to test this behavior because tests are probably not run in a multi gpu instance. maybe the tf.strategy can context can be mocked somehow just for testing?

@owenvallis thanks for clarifying (in the issue thread), indeed the forward pass does the (za,zb) (zb,za). personally I think its better to have the whole loss computation in a single place, its a bit less confusing, it looks exactly like the paper loss and its easier (for me :) ) to reason about the whole averaging over batch size across gpus.
Its up to you, if you prefer to call it in the forward pass Ill update the loss, if you prefer to call it through the loss ill update the forward pass.
Im still not sure why the scaling and margin is needed, the original implementations doesn't do it. Did you see better results using it??

@yonigottesman you raise a good point. The motivation for handling this in the forward pass was to provide support for the different contrastive losses. SimSiam requires 4 inputs, both sets of projector and predictor outputs, while barlow twins only requires a single call using za, zb.
I suppose we could have Simclr and Barlow Twins accept za and zb, and then use the conditional check for Simsiam to handle the 4 input case. I'll take a look at it this weekend.


loginctl attach fails with "Could not attach device: Read-only file system" message

Describe the bug
A modern way to configure multiseat systems seems to is to use systemd's loginctl.
https://wiki.debian.org/Multi_Seat_Debian_HOWTO
But if I try creating a new seat by attaching a GPU to it, I get an error.
To Reproduce
Steps to reproduce the behavior:
Issue following command (I guess any peripherial device present in your /sys will go) :
% sudo  loginctl attach seat-right  /sys/devices/pci0000:00/0000:00:02.0/0000:4b:00.0/drm/card0
Could not attach device: Read-only file system

Expected behavior
A new seat named seat-right should be created, it should be visible with
loginctl list-seats
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
I did some quick investivation, traces lead to https://github.com/systemd/systemd/blob/9494b39dac37894628cf5ef405e6bbc18f69bba9/src/login/logind-dbus.c#L1374
Looks like systemd tries to create a custom udev rule on static etc.
Notify maintainers
@flokli @andir
Metadata
Its basically a nixos-20.03 from channels plus a few additional commits adding some packages on top of it.
nix-shell -p nix-info --run "nix-info -m"                                                                                ~/src/systemd2
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.35, NixOS, 20.03.git.c9d6f28b852 (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3`
 - channels(root): `""`
 - channels(test): `""`
 - channels(dvyal): `""`
 - nixpkgs: `/home/dvyal/nixpkgs`

Maintainer information:
# a list of nixos modules affected by the problem
module: systemd

Nice find! Of course, /etc/udev/rules.d is read-only on NixOS.
Even on non-NixOS, these rules might keep lingering around on the disk on a reboot. Is that intended behaviour?
If it's not, I wonder if it would make sense to have udev load rules from /run/udev/rules.d (or another, ephemeral location) as well.
Care to open a discussion upstream?

I think it's ok for these rules to be persistent. Basically you arrange your input devices and gpus to different seats once and then work for long time in this setting.

Care to open a discussion upstream?
You mean persistent vs ephemeral? I guess It's a question of usability and I have no experience with multiseat.

BTW, in the meanwhile I tried a dirty hack of remounting /nix/store rw. To my surprise, problem persists.

Warning: my multiseat knowledge is almost 10 years old and comes from a time before loginctl and seatctl were considered usable for the most part... everything was managed through xorg.conf and kdm/gdm configuration files.
@akamaus you're right in that most people probably want persistent settings. On NixOS ideally we would want to declaratively configure multiseat through configuration.nix instead of imperatively through loginctl, but it would be nice if logintl still worked (though the results were ephemeral).

@aanderse, agree, combination of declarative and imperative approach is preferable, like we have with users.

@aanderse  I tried implementing multiseat old way, without loginctl, using separate xorg.conf files. Looks like it's no longer possible, If I just start several X servers on different gpus only one is active at the moment (I can switch using ctrl-alt-Fx), moreover, keyboards and mouses all work with currently active server, although I added  InputDevice sections with specific Device options.
I tried setting -seat option for X, this way servers render pictures simultaneously, but one with nondefault -seat can't access input devices.
[ 78290.037] (**) Option "CorePointer"
[ 78290.037] (**) mouse-right: always reports core events
[ 78290.037] (**) evdev: mouse-right: Device: "/dev/input/mouse1"
[ 78290.037] (EE) evdev: mouse-right: Unable to query fd: Inappropriate ioctl for device
[ 78290.056] (EE) PreInit returned 2 for "mouse-right"
[ 78290.056] (II) UnloadModule: "evdev"
[ 78290.056] (II) Using input driver 'evdev' for 'keyboard-right'
[ 78290.056] (EE) systemd-logind: failed to take device /dev/input/by-id/usb-SIGMACHIP_USB_Keyboard-event-kbd: Operation not permitted
[ 78290.056] (**) Option "CoreKeyboard"
[ 78290.056] (**) keyboard-right: always reports core events
[ 78290.056] (**) evdev: keyboard-right: Device: "/dev/input/by-id/usb-SIGMACHIP_USB_Keyboard-event-kbd"
[ 78290.072] (--) evdev: keyboard-right: Vendor 0x1c4f Product 0x26
[ 78290.072] (--) evdev: keyboard-right: Found keys
[ 78290.072] (II) evdev: keyboard-right: Configuring as keyboard
[ 78290.072] (II) XINPUT: Adding extended input device "keyboard-right" (type: KEYBOARD, id 6)

@akamaus to get to this functionality, you probably need loginctl cooperation to tag various devices to various seats (scroll up 4 lines from your link to https://github.com/systemd/systemd/blob/9494b39dac37894628cf5ef405e6bbc18f69bba9/src/login/logind-dbus.c#L1374).
/run/udev/rules.d is also a valid location for ephemeral udev rules. I'd guess upstream made a deliberate choice on having these persisted across rebuilds.
We have all NixOS config-generated udev rules in /etc/udev/rules.d and don't really have a separate place for "rules created imperatively that should survive reboots". We might want to have, but that's another topic too broad for the scope of this issue.
So my proposal here: What about patching logind-dbus.c to put rules in /run/udev/rules.d and warn users to persist these rules in their NixOS configuration, by printing a message when running `loginctl attach?

@flokli your suggestion works indeed. I applied the following patch to systemd
--- a/pkgs/os-specific/linux/systemd/default.nix
+++ b/pkgs/os-specific/linux/systemd/default.nix
@@ -169,6 +169,8 @@ in stdenv.mkDerivation {

     substituteInPlace src/journal/catalog.c \
       --replace /usr/lib/systemd/catalog/ $out/lib/systemd/catalog/
+
+    substituteInPlace src/login/logind-dbus.c --replace /etc/udev/rules.d /run/udev/rules.d
   '';

   # These defines are overridden by CFLAGS and would trigger annoying

Unfortunately, it triggered MASS rebuild, so I just issued nix-build -A systemd and manually replaced systemd-logind binary. Ugly, but loginctl attach now works, rules appear in /var/run/udev/rules.d, as expected.
Settings seem to be active without a reboot, I was able to attach gpu, mouse and keyboard to new seat and X server launched with X :1 --seat new_seat detected them all.
What else have to be done to merge it?

@akamaus you could have set a custom systemd via systemd.package to try this out, so it wouldn't trigger a mass rebuild, but your approach obviously also worked, even though it's a bit more hacky ;-)
When it comes to "how to merge it":
The best way to fix this long-term would be to upstream a fix. I'm not sure if /etc/udev/rules.d would be writable in NixOS any time soon, nor should we just silently add a patch changing the path to some location that's wiped after a reboot.
IIRC, systemd already has some graceful fallbacks if / is mounted read-only. What about filing a PR to systemd, which falls back to /run/udev/rules.d (and emits a warning changes will be lost after a reboot) if it fails to create the rule?
I assume you want to

Still relevant. @Mic92 I could be wrong but were you mentioning something somewhat related to this in #nixos-systemd the other day? Do you have a PR open with upstream?

Just ran into this. Looks like upstream is prepared to take the patch as a fall back one (i.e., if it was modified to first try /etc/udev/rules.d and fall back to /run/udev/rules.d).

Just ran into this. Looks like upstream is prepared to take the patch as a fall back one (i.e., if it was modified to first try /etc/udev/rules.d and fall back to /run/udev/rules.d).

Is this still being worked on? I intend to use multi-seat with NixOS. If it is, in the mean time is there a temporary solution to this?

I also intended to use multiseat on NixOS but I struggle with services.udev.extraRules. It would be nice if there was a way to use loginctl attach.

Still no news?


Why the PhysicalPlanner is an async trait ?

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to  the what)
I have a question for the PhysicalPlanner, looks like in current DataFusion implement, the logical plan creation and optimization is the normal sync functions, but PhysicalPlanner is async, why ?
#[async_trait]
pub trait PhysicalPlanner: Send + Sync {
    /// Create a physical plan from a logical plan
    async fn create_physical_plan(
        &self,
        logical_plan: &LogicalPlan,
        session_state: &SessionState,
    ) -> Result<Arc<dyn ExecutionPlan>>;
}

Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

The scan in TableProvider which creates a physical scan plan from a logical table scan is async. It needs to list partitions and gather statistics that are async.

Thanks a lot for the explanation.


Remove sleep statement from webhook test

Description
Sleep statement increases the test runtime significantly and does not have an obvious use.
Changes proposed in this pull request:

Remove sleep statement from webhook test

Related issue(s)

@skhalash would like to run the unit tests on Prow a few times and ensure that it is not introducing any flakiness before merging.

/test pull-telemetry-manager-unit-test

/test pull-telemetry-manager-unit-test

/test pull-telemetry-manager-unit-test

/test pull-telemetry-manager-unit-test

/unhold