Travis/build stages & fixes

use travis-ci build stages for compiling on various dmd,gdc, and ldc versions on both Mac and Linux, then a separate stage for running tests against database.
you'll get build structured like this

https://travis-ci.org/SingingBush/mysql-native/builds/279178962
if you don't want to merge this PR please at least cherry pick 1964b96 and update the compiler versions in the current travis file. Also, vibe-d v8 has been out a while now so I dropped the v7

lgtm

Thanks very much for this PR. Before I merge, I have some questions about a few things. I will ask as in-line github comments...

Ok, there's several things in here I definitly want to merge - fixing the libevent dependency for travis is a big one (I admit, the broken test builds from that have gone on too long), adding tests for osx, the std.datetime fixes.
The travis build stages thing though, I've been looking closer into, and I'm not so sold on using that to separate build from test. What that change means is:

None of the tests will be run at all unless it successfully compiles on ALL compilers. So when a build failure occurs on one compiler version, then that'll be a problem as it will automatically block all testing on all other compilers.

It drastically increases the time used by travis, as the "Run tests" stage still has to recompile again anyway (due to travis's task isolation).

It drastically bloats the .travis.yml file and introduces a lot of copy-pasted non-DRYness. Even if I move the before_script stuff to its own external script, that's still a lot of non-DRY.

If you can undo the travis build stages stuff (but keeping the libevent fixes and osx builds), add back in the compiler versions that were removed, and revert the changes to dub.selections.json and source/mysql/test/integration.d (as I mentioned in a separate comment), then I'd very much like to merge the rest.

I'm happy to do another PR that is without build stages however it's not possible to keep supporting all the compiler versions currently used while also adding support for the newer front ends.
I'm currently using dmd 2.076, if I run dub test -c unittest-vibe with the project now (with prepared.d fixed) I get the following compile error:
vibe-d-0.7.29/vibe-d/source/vibe/core/core.d(1078,18): Error: function vibe.core.core.CoreTask.state cannot override final function core.thread.Fiber.state

If the vibe-d dependency is updated to v8 then it won't be able to support the older compilers (vibe-d 8 requires at least dmd 2.071.2).
Is it essential to support dmd all the way back to 2015 (when 2.068.2 was released)? If it is required then perhaps it would be best to have a branch that focuses on the older compiler versions and vibe-d 7 while another branch is used for more current compiler versions and vibe-d 8

On September 29, 2017 6:36:03 PM EDT, Samael<EMAIL_ADDRESS>wrote:

I'm happy to do another PR that is without build stages however it's
not possible to keep supporting all the compiler versions currently
used while also adding support for the newer front ends.
I'm currently using dmd 2.076, if I run dub test -c unittest-vibe
with the project now (with prepared.d fixed) I get the following
compile error:
vibe-d-0.7.29/vibe-d/source/vibe/core/core.d(1078,18): Error: function
vibe.core.core.CoreTask.state cannot override final function
core.thread.Fiber.state

If the vibe-d dependency is updated to v8 then it won't be able to
support the older compilers (vibe-d 8 requires at least dmd 2.071.2).
Is it essential to support dmd all the way back to 2015 (when 2.068.2
was released)? If it is required then perhaps it would be best to have
a branch that focuses on the older compiler versions and vibe-d 7 while
another branch is used for more current compiler versions and vibe-d 8

Ah, I see, I didn't realize there was a compile error with that version of vibed and dmd 2.076. Latest dmd definitely needs to be supported, but it looks like the vibe 0.7.x line does support it as of 0.7.32.
So we can set 0.7.32 as the minimum vibe, put the older compiler versions back in, and if any of those older compilers error out with 0.7.32 I can look into either fixing or otherwise finally deprecating the compilers as-needed.

I've started the changes as a new branch off master. See the build results, there is a problem with dmd 2.074.1 and ldc 1.4.0:
ource/mysql/test/integration.d(1119): Error: template std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals cannot deduce function from argument types !()(int), candidates are:

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2212):        std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref const(typeof(this)) rhs)

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2222):        std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref const(T) rhs)

source/mysql/test/integration.d(1129): Error: template std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals cannot deduce function from argument types !()(int), candidates are:

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2212):        std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref const(typeof(this)) rhs)

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2222):        std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref const(T) rhs)

that was the reason I'd made those changes to integration.d
If you are happy with the changes there I will squash all the travis related changes into a single commit and do a new PR.
It may be worth just setting allow failures for dmd 2.074.1 and ldc 1.4.0

On September 30, 2017 7:30:03 AM EDT, Samael<EMAIL_ADDRESS>wrote:

I've started the changes as a new
branch
off master. See the build
results,
there is a problem with dmd 2.074.1 and ldc 1.4.0:
ource/mysql/test/integration.d(1119): Error: template
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals cannot deduce
function from argument types !()(int), candidates are:

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2212):
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref
const(typeof(this)) rhs)

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2222):
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref
const(T) rhs)

source/mysql/test/integration.d(1129): Error: template
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals cannot deduce
function from argument types !()(int), candidates are:

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2212):
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref
const(typeof(this)) rhs)

/home/travis/dlang/ldc-1.4.0/bin/../import/std/typecons.d(2222):
std.typecons.Nullable!(VariantN!32LU).Nullable.opEquals()(auto ref
const(T) rhs)

that was the reason I'd made those
changes
to integration.d

Ah, ok, I see. Apparently this is due to: https://issues.dlang.org/show_bug.cgi?id=17482
(I had completely forgotten about that.)
Don't worry about that, I bet it can be worked around pretty easily. And if not, then like you say, tossing in an allow failure for that compiler version is certainly an option.

If you are happy with the changes there I will squash all the travis
related changes into a single commit and do a new PR.

Yes, looks excellent. :) Don't bother squashing though: The separate commits help me follow the various changes involved, so I'd like to keep that.

closing as #135 was merged instead


[spree] Not showing the product list

Hello sir,
I am using spree 4.4 , spree nextjs-commerce (https://github.com/spree/nextjs-commerce) complied successfully. On browsing localhost:3000 front page loads successfully but products in menus are not showing and giving an error below
event - compiled client and server successfully in 4.4s (793 modules)
wait  - compiling / (client and server)...
event - compiled client and server successfully in 1545 ms (888 modules)
getAllProducts called. Configuration:  getAllProductsVariables:  { first: 6 } config:  { locale: 'en-US', locales: [ 'en-US', 'es' ] }
apiFetch called. query =  url =  __UNUSED__ queryData =  {
  variables: { methodPath: 'products.list', arguments: [ {}, [Object] ] }
} fetchOptions =  {}
Looking for products.list in Spree Sdk.
Checking part products.
Calling the Spree API: GET https://demo.spreecommerce.org/api/v2/storefront/products?include=primary_variant%2Cvariants%2Cimages%2Coption_types%2Cvariants.option_values&per_page=6&image_transformation%5Bquality%5D=100&image_transformation%5Bsize%5D=1000x1000
getAllPages called. Configuration:  query:  undefined userConfig:  { locale: 'en-US', locales: [ 'en-US', 'es' ] } preview:  undefined url:  undefined
apiFetch called. query =  url =  __UNUSED__ queryData =  { variables: { methodPath: 'pages.list', arguments: [ [Object] ] } } fetchOptions =  {}
Looking for pages.list in Spree Sdk.
Checking part pages.
Calling the Spree API: GET https://demo.spreecommerce.org/api/v2/storefront/cms_pages?per_page=500&filter%5Blocale_eq%5D=en-US
getSiteInfo called. Configuration:  query:  undefined getSiteInfoVariables  {} config:  { locale: 'en-US', locales: [ 'en-US', 'es' ] }
apiFetch called. query =  url =  __UNUSED__ queryData =  { variables: { methodPath: 'taxons.list', arguments: [ [Object] ] } } fetchOptions =  {}
Looking for taxons.list in Spree Sdk.
Checking part taxons.
Calling the Spree API: GET https://demo.spreecommerce.org/api/v2/storefront/taxons?filter%5Bparent_permalink%5D=categories
apiFetch called. query =  url =  __UNUSED__ queryData =  { variables: { methodPath: 'taxons.list', arguments: [ [Object] ] } } fetchOptions =  {}
Looking for taxons.list in Spree Sdk.
Checking part taxons.
Calling the Spree API: GET https://demo.spreecommerce.org/api/v2/storefront/taxons?filter%5Bparent_permalink%5D=brands

Can anybody help me

resolved by adding cors nginx policies on  the side of spree hosted server
https://enable-cors.org/server_nginx.html
https://docs.viblast.com/player/cors/cors-on-nginx

Hey there! Thank you for your contribution. We have decided to take Next.js Commerce in a new direction and will be closing out current PRs and issues due to this change. Please see this PR for more details: https://github.com/vercel/commerce/pull/966


Revert support for template members

This reverts PR: https://github.com/Semmle/ql/pull/2271
Internal PR: https://git.semmle.com/Semmle/code/pull/35338

Closing this PR. The underlying problem is caused by an extractor bug.


my local keyboard change in to english

I can use y local language keyboard into scrcpy,
my keyboard language in Ubuntu is Persian and need access to this language in android(in android I have Persian keyboard)

This is not supported, and there are many duplicates of this request.
https://github.com/Genymobile/scrcpy/blob/master/FAQ.md#special-characters-do-not-work


Add "scale" for senergy template to support SolarMax SMT series

SolarMax SMT series inverters report PAC multiplied by 10, so the template needs to accept the "scale" parameter to multiplie the PAC value by e. g. 0.1

I've done the change, but am not allowed to push a branch & create a PR then...

You need to clone the repo first


Separate routines from stats / recovery

Right now routines are all mixed with recovery things which is very confusing. Would be better if we had generic stats functions that get triggered from routines and recovery.  Idea is that routines are on cron and not everything is run.  Most things don't need to run as a routine and should in fact be one off actions run as a recovery.
Routines then calls stats functions on cron and recovery the same but one off.
Routines have been ooming and over time, we'll want to write routines that can be scheduled properly.

Separated logic from infinite loops so that logic can be called in recovery and in long running routines. Now routines are either in:

init -> run once on startup
cron -> schedule
recovery -> run once


Parse Cassandra execute statements

Unfortunately this had caused a clippy lint, not sure what the best course of action is:

Leave the lint turned off
Put the big variants in a box (alot of code changes because we will have to dereference all the Cassandra frames after parsing instead of just pattern matching)

I think the answer is Execute(Box<BodyReqExecuteOwned>) right?
Thats what we already do for CassandraOperation::Query and CassandraOperation::Result.

Oh true! If only clippy was smart enough to tell me that. Thanks.


Ruby 2.2.3 openssl error to install native extensions

./project.h:116:10: fatal error: 'openssl/ssl.h' file not found
#include <openssl/ssl.h>

$ rbenv install --version
#=> ruby-build 20151028

$ $ brew list
ansible                 docker                  glew                    libpng                  mongodb                 postgresql              tmux
autoconf                docker-compose          glm                     libtiff                 nginx                   psgrep                  tree
automake                docker-machine          go                      libtool                 node                    rbenv                   unfs3
bash                    elixir                  gource                  libxml2                 openssl                 readline                unixodbc
bash-completion         erlang                  heroku-toolbelt         libxslt                 ossp-uuid               redis                   webp
bazaar                  fig                     icu4c                   libyaml                 pcre                    ruby-build              wget
boost                   freetype                imagemagick             luajit                  pg_upgrade_internal.log sdl2                    wxmac
cabal-install           gettext                 jpeg                    macvim                  phantomjs               sdl2_image              xz
cmake                   ghc                     libevent                memcached               pidof                   sysdig
cscope                  git                     libiconv                mercurial               pkg-config              the_silver_searcher

+1
rbenv install 2.2.3 fails for me as well with the following error :-
ossl_ssl.c:141:27: error: 'SSLv3_method' undeclared here (not in a function)
     OSSL_SSL_METHOD_ENTRY(SSLv3),
                           ^
ossl_ssl.c:119:69: note: in definition of macro 'OSSL_SSL_METHOD_ENTRY'
 #define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
                                                                     ^
ossl_ssl.c:142:27: error: 'SSLv3_server_method' undeclared here (not in a function)
     OSSL_SSL_METHOD_ENTRY(SSLv3_server),
                           ^
ossl_ssl.c:119:69: note: in definition of macro 'OSSL_SSL_METHOD_ENTRY'
 #define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }
                                                                     ^
ossl_ssl.c:143:27: error: 'SSLv3_client_method' undeclared here (not in a function)
     OSSL_SSL_METHOD_ENTRY(SSLv3_client),
                           ^
ossl_ssl.c:119:69: note: in definition of macro 'OSSL_SSL_METHOD_ENTRY'
 #define OSSL_SSL_METHOD_ENTRY(name) { #name, (SSL_METHOD *(*)(void))name##_method }

My ruby-build version -
$ rbenv install --version
#=> ruby-build 20151028

ruby 2.2.3 on my machine was installed
the problem with openssl occurs on eventmachine gem, i solve using this
gem install eventmachine -v '1.0.8' -- --with-cppflags=-I/usr/local/opt/openssl/include

I would argue that this is a problem with eventmachine gem build process. It seems that Ruby was correctly installed and its openssl extension was linked to OpenSSL from Homebrew. Everything was fine, except that eventmachine wanted to link to openssl manually during its own build process (for some reason—I don't know why it didn't use Ruby's extension). You worked around your issue, but it doesn't change the fact that eventmachine's build process is not optimal.

I think you need to add ```--with-openssl-dir=`brew --prefix openssl```` like this.
$ CONFIGURE_OPTS="--with-openssl-dir=`brew --prefix openssl`" rbenv install 2.2.3

homebrew didn't create symlink into /usr/local directory.

this only about 2.2.3. my 2.2.2 installation works fine, why?

@hsbt does not work, same problem

I can reproduce Ruby 2.2.2 and 2.2.3. It's eventmachine or your environment issue.
gem install eventmachine -v '1.0.8' -- --with-cppflags=-I`brew --prefix openssl`/include is correct instruction I think. It causes by homebrew mechanism .

it's eventmachine issue !

Closing since it's not our issue. Thanks for the investigation!


Marks

Closes: #177
Checklist

[x] I have written tests
[ ] I have provided documentation

Changelog

Added schemas for marks and personal marks
Created a repository and service for each of them
Implemented functionality for creating marks, and adding and removing them from users
Added functions for mark expiration calculation

How to test
yarn test

DATE FNS???

DATE FNS???

Wip ;)


Build crash fix: AnyObject -> NSDictionary

As seen in swift crashers while building https://github.com/apple/swift/pull/20103.

@swift-ci please test


Update sqlsrv_sample.php

hi @vaibhavsharma56 , thanks for your contribution, but as I mentioned in #867 and #870 , please merge to our dev branch, not master. At the next preview we will merge to master.

Coverage remained the same at 75.367% when pulling 1eb61664b45390b7b3dbef64a8e6d366df400973 on vaibhavsharma56:patch-1 into 14acf0007323e3b54fda89ad0c4c5c789ad9545d on Microsoft:master.


Using the FN Key on the keyboard shortcut manager

Description of the new feature / enhancement
Visible action when FN key is pressed so that a combination of i.e. "FN+2" or "FN+Alt gr+H" trigger a sequence of keystrokes or even more complicated shortcuts.
Scenario when this would be used?
i.e.
In games for shortcuts in heated battles :)
For inserting your signature in Outlook when composing an Email (Alt+i+t+s+Enter)
and so on
Supporting information
I tried using the FN key in the keyboard shortcut manager but nothing happened.
I am using the Logitech PRO X TKL. Its Logitech Software functions are not really helpful since you can only combine the FN key with the F keys above the row with the digits which is NOT very handy!

Fn key itself tends to be a hardware level key and we can't detect it. Things like Fn+F12, that can trigger different things and actually the driver executes it. We're looking at maybe scancodes to override this but sadly, some stuff at our level, just can't do. /helpeduser

Okay...That's a real shame! Thanks for the explanation! :)


Update bazel to 4.0.0

What this PR does / why we need it:
The next PRs will update rules_go, rules_docker as well.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

🚀
/lgtm

/approve


Should we add More information items under Profiles, Topologies and Market models menus?

As suggested in https://github.com/quintel/etmoses/issues/895, I would prefer to make have a More information item under the Local Energy Solutionsmenu that redirects to the ETMoses Wiki. That way, we can easily update the documentation regarding the Local Energy Solutions and avoid having to store the same information in multiple places.
Likewise I would suggest to add More information items under all other menus where

Profiles -> More information would link to https://github.com/quintel/etmoses/wiki/Profiles
Topologies -> More information would link to https://github.com/quintel/etmoses/wiki/Topology
Market models -> More information would link to https://github.com/quintel/etmoses/wiki/Market-models

What do you think,  @MarjoleinSchwachofer , @ChaelKruip , @dennisschoenmakers , @antw and @AlexanderWirtz ?

Part of https://github.com/quintel/etmoses/issues/893.

Assigning @grdw to do this.

Assigning @grdw .

This has been implemented on the Beta-server. Thanks, @grdw !


[home-assistant] Prometheus integration is not enabled when metrics are enabled

Details
Helm chart name and version:
home-assistant 11.0.4
Container name and tag:
homeassistant/home-assistant:2021.6.3
Relevant Helm values:
  metrics:
    enabled: true
    serviceMonitor:
      bearerTokenSecret:
        name: home-assistant-prometheus-access-token
        key: token

What did you expect to happen:
When metrics setting is enabled I assumed it would just work, but prometheus kept alerting - api/prometheus returned 404. Apparently one needs to enable prometheus integration for home-assistant in its settings (/config/configuration.yaml). See https://www.home-assistant.io/integrations/prometheus/
E.g.:
prometheus:
  namespace: hass

It would be nice, if this happened automagically - enabling metrics, would enable the integration.

Having a note about this in the docs would be quite helpful.


bazel build //tensorflow/examples/android:tensorflow_demo

when i run  bazel build //tensorflow/examples/android:tensorflow_demo.i get the result as follow:
INFO: Found 1 target...
ERROR: missing input file '@androidsdk//:build-tools/24.0.3/lib/dx.jar'.
ERROR: /home/zhu/tensorflow/tensorflow/examples/android/BUILD:63:1: //tensorflow/examples/android:tensorflow_demo: missing input file '@androidsdk//:build-tools/24.0.3/lib/dx.jar'.
Target //tensorflow/examples/android:tensorflow_demo failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/zhu/tensorflow/tensorflow/examples/android/BUILD:63:1 1 input file(s) do not exist.
INFO: Elapsed time: 18.794s, Critical Path: 0.60s

You may did a mistake in this step https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android#edit-workspace
Did you provide the right payh?

This issue is automatically closed due to lack of activity. Please re-open if this is still an issue for you. Thanks!


[Backport 2.x] Switch spacetype similarity function to MAXIMUM_INNER_PRODUCT

Signed-off-by: John Mazanec<EMAIL_ADDRESS>(cherry picked from commit 62fc890e068238bd63dac713d989c4e4272028ba)
Description
Backport #1532 to 2.x manually
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Failing with unrelated flaky test: #1522


Run spark-hdfs connector on Travis

Spark Connector spark-local is running on Travis! This covers majority of possible errors for all sparkcore dependent connectors


GPS python send message to Azure iot from Linux device

Hi guys,
from Linux with python I can send messages to Iot Azure with the code below,
but could you also extend on how to pass with it GPS, please?
import os
    import asyncio
    from azure.iot.device.aio import IoTHubDeviceClient
etc

from simple_send_message from python azure sdk
simple_send_message.py
Sending message...
Message successfully sent!

@AndreV84 can you specify what you mean by GPS?

@AndreV84, in #772 you mentioned wanting to use paho-mqtt. Does this mean you wish to directly use an MQTT broker? Or do you want to use the client (recommended).

@cartertinney These are two separate issues. There is a reason why they are separate.
Issue 772 is for procurement of paho-mqtt example, if you are able to supply it. It would be kind of solved once there is paho-mqtt that sends a test message to Azure Iot
issue 771 is for ingestion of GPS data into simple_sample_message.py of the given azure-iot-sdk-python github project. It can be considered solved once ingestion of GPS data from device to the Iot server is working
under GPS data I mean sensor information from GPS Unit connected to Device so that it gets recieved by IoT Azure e.g. IoT Azure MQTT

@cartertinney
are you able to provide a solution for former issue ? for the latter? for both? for neither of the two?

@AndreV84 can you specify what you mean by "pass it with GPS"?

https://en.wikipedia.org/wiki/Global_Positioning_System

@cartertinney you changed the original text in your question so it got  a reverse meaning "pass it with GPS" versus the original meaning of passing the GPS with the code

@AndreV84
Is there a reason that the current message functionality cannot support GPS data?
This sample should serve your purposes unless there is something I am missing about GPS data that makes it unique - can you not put the GPS data in the body of a message as a string?
https://github.com/Azure/azure-iot-sdk-python/blob/master/azure-iot-device/samples/async-hub-scenarios/send_message.py

@cartertinney Thank you for following up.

To your other issue, I do not understand what you mean by "paho-mqtt example". The azure-iot-device SDK is built on top of Paho - by using the IoTHubDeviceClient you will be able to communicate between a device and the IoTHub. Is this not sufficient? Are you asking for tools to directly use Paho in lieu of a using the IoTHubDeviceClient?

While paho-mqtt example is somewhat transparent for reading/customization
the azure-iot python SDK is build of the top of it, so it doesn't allow to access it or read its function directly as it is implemented at hidden layer that would add more complexity.
So with paho-mqtt for minor modificatin it would be required to read/edit code of one file
for doing with azure python sdk it will require to read/ edit multiple files probably, at least it becomes a level of magnitude more difficult, in my opinion.
I saw that there were Azure paho-mqtt examples published previously. Is there an example with plain paho-mqtt that works for sending a simple message [hello world for example] to the Azure Iot without adding an extra layer of complexity in a form of necessity to get through the source code of the azure-iot-python-sdk module? It becomes two issues instead of one as long as not only the single transparent python file of the example needs to be read, but also the hidden sdk structure needs to be reverse engineered or digged out from somewhere to address probable sdk constraints, in my opinion.However, I request to share a working paho-mqtt example because it will allow to address a complexity of one small text file to get a message sent to Iot from linux device instead of addressing the complexity of both the small text file with example but also with the complexity of the entire azure-cli-python-sdk. I

can we with azure-client-python sdk use in the sentence the following construction like we used in golang?
`package main
...
func main() {
//setup mqtt, all hard coded
topic := "demo"
broker := "tcp://localhost:1883"
id := "myclient"
qos := 1
opts := MQTT.NewClientOptions()
opts.AddBroker(broker)
opts.SetClientID(id)
...

`
Otherwise how exactly to forward the pipeline from connected GPS device so that the azure-sdk-pythin reads it from linux device like /dev/gps0? from web socket? I am looking for an example or demo that shows how to read from GPS transmitting it to IoT
Finally the reason why the issue 772 was a separate issue is because the issue 772 solution would be to find out how to send  a simple message like in your example but with paho-mqtt directly
m = Message("<GPS data goes here>")
await device_client.send_message(m)

While the solution for the issue 771 that you pointed out only shows how to send a simple message.
While what needs to be addressed in order for the solution to get implemented  for 771 is the import structure that will read from GPS device to the python azure-iot-sdk sample, so that it will read also transmit continiously. It will be solved once the continious flow of data appears at Azure Iot that comes from connected to USB or UART GPS unit. Where exactly/ how exactly to put the usb device outputs or uart outputs in a way they will get transmitted by the azure-python-sdk it is the questino in 771, not how to send a simple message, as I have already pointed out I was able to send just a message. But it needs to be either sending tcp socket or structured message sequence.

@cartertinney so fo r 772 could you provide a confirmed to work example with direct mqtt, please
probable examples would look like the above but would be confirmed to work at all
from paho.mqtt import client as mqtt

def on_connect(client, userdata, flags, rc):
    print "Connected with result code: %s" % rc
    client.subscribe("devices/<YOUR DEVICE ID>/messages/devicebound/#")

def on_disconnect(client, userdata, rc):
    print "Disconnected with result code: %s" % rc

def on_message(client, userdata, msg):
    print " - ".join((msg.topic, str(msg.payload)))
    # Do this only if you want to send a reply message every time you receive one
    client.publish("devices/<YOUR DEVICE ID>/messages/events", "REPLY", qos=1)

def on_publish(client, userdata, mid):
    print "Sent message"

client = mqtt.Client(cleint_id=<YOUR DEVICE ID>, protocol=mqtt.MQTTv311)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message
client.on_publish = on_publish
client.username_pw_set(username="<YOUR NAMESPACE>.azure-devices.net/<YOUR DEVICE ID>",
                       password="<YOUR SHARED ACCESS SIGNATURE FOR THE DEVICE>")
client.tls_insecure_set(True) # You can also set the proper certificate using client.tls_set()
client.connect("<YOUR NAMESPACE>.azure-devices.net", port=8883)
client.loop_forever()

source https://stackoverflow.com/questions/35452072/python-mqtt-connection-to-azure-iot-hub
or like
from paho.mqtt import client as mqtt
import time
import ssl

def on_subscribe(client, userdata, mid, granted_qos):
print('Subscribed for m' + str(mid))

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

def on_message(client, userdata, message):
    print("Received message '" + str(message.payload) + "' on topic '" + message.topic + "' with QoS " + str(message.qos))

def on_log(client, userdata, level, buf):
    print("log: ",buf)

device_id = "device1"
iot_hub_name = "myIoTHub"
sas_token = "SharedAccessSignature sr=myIoTHub.azure-devices.net%2Fdevices%2Fdevice1&sig=****&se=1586926815"
client = mqtt.Client(client_id=device_id, protocol=mqtt.MQTTv311,  clean_session=False)
client.on_log = on_log
client.tls_set_context(context=None)

# Set up client credentials
username = "{}.azure-devices.net/{}/api-version=2018-06-30".format(iot_hub_name, device_id)
client.username_pw_set(username=username, password=sas_token)

# Connect to the Azure IoT Hub
client.on_connect = on_connect
client.connect(iot_hub_name+".azure-devices.net", port=8883)

# Publish
client.publish("devices/{device_id}/messages/events/".format(device_id=device_id), payload="{}", qos=0, retain=False)

# Subscribing on the topic ,
client.on_message = on_message
client.on_subscribe = on_subscribe
client.subscribe("devices/{device_id}/messages/devicebound/#".format(device_id=device_id))
client.subscribe("$iothub/twin/PATCH/properties/desired/#")
client.subscribe("$iothub/methods/POST/#")

client.loop_forever()

source https://stackoverflow.com/questions/57243449/connecting-with-m2mqtt-paho-python-client-to-azure-iot-hub/66741952#66741952

However fo 771 the answer would address explicitly how to load from GPS device, but passing the message not by mqtt directly, but by python-iot-sdk, so ther solutions like the above that use directly mqtt wouldn't be addressing the the concerns of 771 while would address the concern of 772.
For 772 the solution confirmed to work or supported would look like
#! /usr/bin/python
# Based on
# Written by Dan Mandle http://dan.mandle.me September 2012
# License: GPL 2.0

import os
from gps import *
from time import *
import time
import threading
import json
import paho.mqtt.client as paho

gpsd = None

class GpsPoller(threading.Thread):
  def __init__(self):
    threading.Thread.__init__(self)
    global gpsd #bring it in scope
    gpsd = gps(mode=WATCH_ENABLE) #starting the stream of info
    self.current_value = None
    self.running = True #setting the thread running to true

  def run(self):
    global gpsd
    while gpsp.running:
      gpsd.next() #this will continue to loop and grab EACH set of gpsd info to clear the buffer

if __name__ == '__main__':
  gpsp = GpsPoller() # create the thread
  try:
    gpsp.start() # start it up
    while True:
      if gpsd.fix.latitude > 0:
        row = [ { 'latitude': str(gpsd.fix.latitude),
         'longitude': str(gpsd.fix.longitude),
         'utc': str(gpsd.utc),
         'time':   str(gpsd.fix.time),
         'altitude': str(gpsd.fix.altitude),
         'eps': str(gpsd.fix.eps),
         'epx': str(gpsd.fix.epx),
         'epv': str(gpsd.fix.epv),
         'ept': str(gpsd.fix.ept),
         'speed': str(gpsd.fix.speed),
         'climb': str(gpsd.fix.climb),
         'track': str(gpsd.fix.track),
         'mode': str(gpsd.fix.mode)} ]

        json_string = json.dumps(row)
        client = paho.Client()
        client.username_pw_set("dfdf","34343434")
        client.connect("cloudqx.com", 14162, 60)
        client.publish("rpiwzgps", payload=json_string, qos=0, retain=True)

        time.sleep(60)

  except (KeyboardInterrupt, SystemExit): #when you press ctrl+c
    gpsp.running = False
    gpsp.join() # wait for the thread to finish what it's doing

source https://github.com/tspannhw/rpizw-nifi-mqtt-gps/blob/master/gps2.py
but it would need to be either confirmed to work or supported at all solution.
Are you able to provide a solution for 771 issue? for 772? both? neither of the two?

lets rephrase 771.
how to add azure iot sdk components into the code above so it actually not only takes the gps data [ if it works at all], but also sends it to Azure Iot?
for 772:
how to pass a simple text message like hello world with paho mqtt/mosquitto directly to Azure Iot? without loading the sdk at all?

@AndreV84
With regard to 772, this is something we are actually currently considering internally. We do not currently have a set of tools to empower you to directly use Paho and cut out the client layer, but this is not the first time we have been asked for it, and we will take this as a useful data point which can direct our efforts going forth.
While unfortunately, I do not have tools or official samples to share with you on this matter, I can provide you with this guide which describes exactly how to directly interface with the MQTT protocol on Paho. I can confirm to you that the information in here is all correct, as this is exactly how we use Paho in our client layer. The tricky part is going to be dealing with authentication. This guide describes the technical spec of how to generate a SAS token but unfortunately only provides an example in Node JS, however the logic is simple enough that you shouldn't have much trouble moving it over to Python.
Like I said, we're looking at trying to make this process more user-friendly, and we are working on some things that will make this easier in the future, but right now they are not official releases.
With regard to 771 specifically, I still am not quite clear on what the issue with using GPS and the client together specifically are. From what I see in your examples, it seems like you could simply just use the IoTHubDeviceClient to do send message operations every time you poll your GPS and receive data. Is there a reason you are unable to do this? If so, can you please specify (ignoring the other 772 issue)

@cartertinney
Thank you for following up
you wrote: "If you provide me with working example I will adopt it so it works with my device/GPS unit."
But unless there is a working example provided that does send GPS data to IoT  with your sdk it is just complicated,
Regarding the other issue - there was paho-mqtt example. NMoreover I have SAS. I just need a working examplew previously supplied by Azure via github but working with current releases

@AndreV84
I don't know anything about a previous paho-mqtt sample, as far as I know we have never had one, as this is not a feature we have ever supported tooling for. However, as mentioned this guide describes how to do exactly what you want. If this guide is not sufficient, please specify exactly what the issue is, and we may be able to provide additional support.
client_id = "<client id as specified in guide>"
username = "<username as specified in guide>"
password = "<password as specified in guide>"
hostname = "<IoTHub hostname>"
topic = "<topic as specified in guide>"     # if you want to send to multiple diff topics might be best to define a fn
payload = "<your data>"

client = paho.Client(client_id=client_id, clean_session=False, protocol=mqtt.MQTTv311)
client.username_pw_set(username=username, password=password)
client.connect(hostname=hostname, port=8883, keepalive=60)
client.publish(topic=topic, payload=payload, qos=1)

Again, as far as your GPS concerns, I'm still unclear what the problem is. You say that it is complicated, but haven't specified how. Can you clarify exactly what you are trying to do with GPS and why the IoTHubDeviceClient is not sufficient for that purpose? I need more information in order to assist you.

Thank you for following up
you wrote: ", but haven't specified how"
I responded - I need a working example so I could adopt it.

thank you for sharing the code above I will try to put devices values into it to see if it sends a test message
but probably the part that sends a test message is not in the code above that is shared

@AndreV84 We Our samples are all general purpose examples that are generic. We do not have a GPS-specific example because nothing about the using the client should be unique to using a GPS as far as I can tell.
Any data you want to send to an Azure IoTHub should be sent in a Message via the send_message() method. You should be able to send any kind of data that way - it doesn't matter what the contents are. You can pull your GPS location information from you GPS module, and then format the data as a string, and send it to the Hub as a Message.
If you are trying to do something else, please describe what it is you want to do because I don't understand what the problem is.

Specifically,
given there is GPS output on /dev/pts/9
how to modify the code below  so it will transmit it?
# Copyright (c) Microsoft. All rights reserved.
# Licensed under the MIT license. See LICENSE file in the project root for full license information.
import sys
import time
import serial

# Using the Python Device SDK for IoT Hub:
# https://github.com/Azure/azure-iot-sdk-python
# The sample connects to a device-specific MQTT endpoint on your IoT Hub.
from azure.iot.device import IoTHubDeviceClient, Message

# Settings for reading from Arduino Serial
serialPort= "/dev/pts/10" #Change it to your Serial Port, Check in Arudino IDE
baudRate = 9600
ser = serial.Serial(serialPort, baudRate, timeout=0.5)

# The device connection string to authenticate the device with your IoT hub.
# Using the Azure CLI:
# az iot hub device-identity show-connection-string --hub-name {YourIoTHubName} --device-id MyNodeDevice --output table
CONNECTION_STRING="--------------------------------------------------"

# Define the JSON message to send to IoT Hub.
MSG_TXT = '{{"Light": {light}}}'

def iothub_client_init():
# Create an IoT Hub client
    client = IoTHubDeviceClient.create_from_connection_string(CONNECTION_STRING)
    return client

def iothub_client_telemetry_sample_run():

    try:
        client = iothub_client_init()
        print ( "IoT Hub device sending periodic messages, press Ctrl-C to exit" )

        while True:
# Build the message with simulated telemetry values.
            time.sleep(0.1)
            light = ser.readline().decode("UTF-8")[:-2]
            if light:
                msg_txt_formatted = MSG_TXT.format(light=light)
                message = Message(msg_txt_formatted)

# Send the message.
        print( "Sending message: {}".format(message) )
        client.send_message(message)
        print ( "Message successfully sent" )
        time.sleep(1)

except KeyboardInterrupt:
    print ( "IoTHubClient sample stopped" )

ser.close()
if name == 'main':
print ( "IoT Hub Quickstart #1 - Simulated device" )
print ( "Press Ctrl-C to exit" )
iothub_client_telemetry_sample_run()
the error output is

python3.9 send.py
IoT Hub Quickstart #1 - Simulated device
Press Ctrl-C to exit
IoT Hub device sending periodic messages, press Ctrl-C to exit
Traceback (most recent call last):
File "/home/nvidia/send.py", line 59, in
iothub_client_telemetry_sample_run()
File "/home/nvidia/send.py", line 40, in iothub_client_telemetry_sample_run
light = ser.readline().decode("UTF-8")[:-2]
File "/home/nvidia/.local/lib/python3.9/site-packages/serial/serialposix.py", line 560, in read

plain output of the serial looks like that
 cat /dev/pts/9
$GPGGA,170607,3803.5483,N,02400.8192,E,1,04,5.6,1974.4,M,34.5,M,,*79
$GPRMC,170607,A,3803.5483,N,02400.8192,E,2093.4,308.0,010621,5,E,A*32
$GPGGA,170609,3803.7779,N,02359.4016,E,1,04,5.6,1987.7,M,34.5,M,,*76

@AndreV84
This is an error in your serialized reading. The error is coming from the serial library you are using. As you can see in your log, the error is coming from line 560 of a file in the package called serialposix.py when you try to read the value out on line 40 of your sample.
I have no insight into serial. That is a completely different product that has nothing to do with the azure-iot-device package - this error is occurring before you pass any values to our client.

so GPS transmission is not supported by the python  azure sdk ? is there any working example of how to implement it either with paho mqtt or with client python sdk? with former? latter? both? neither of the two?

I can read the data with just these lines;
but how to transmit it with the azure sdk?
import sys

with open("/dev/pts/26", "wb+", buffering=0) as term:
        while True:
        	print(term.read(26).decode(), end='')
        sys.stdout.flush()

@cartertinney Thank you very much
let me try with it!
nvidia@agx:~$ cat pts_v2.py
import sys

with open("/dev/pts/15", "wb+", buffering=0) as term:
        while True:
        	print(term.read(15).decode(), end='')
        sys.stdout.flush()
nvidia@agx:~$ python3 pts_v2.py
$GPGGA,100901,3756.2617,N,02340.7683,E,1,04,5.6,809.2,M,34.5,M,,*47
$GPRMC,100901,A,3756.2617,N,02340.7683,E,5849.7,267.1,020621,5,E,A*31
$GPGGA,100903,3749.2298,N,02343.4153,E,1,04,5.6,455.4,M,34.5,M,,*41
$GPRMC,100903,A,3749.2298,N,02343.4153,E,13204.4,163.4,020621,5,E,A*01
$GPGGA,100905,3743.6511,N,02348.5524,E,1,04,5.6,180.8,M,34.5,M,,*40

but
nvidia@agx:~$ cat send.py
import sys
# https://github.com/Azure/azure-iot-sdk-python
# The sample connects to a device-specific MQTT endpoint on your IoT Hub.
from azure.iot.device import IoTHubDeviceClient, Message
device_client = IoTHubDeviceClient.create_from_connection_string("redacted------------------------------redacted--------=")
with open("/dev/pts/15", "wb+", buffering=0) as term:
        while True:
            data = str(term.read(15).decode(), end='')
            message = Message(data)
            device_client.send_message(message)
        sys.stdout.flush()

nvidia@agx:~$ python3 send.py
Traceback (most recent call last):
  File "send.py", line 8, in <module>
    data = str(term.read(15).decode(), end='')
TypeError: 'end' is an invalid keyword argument for this function

at another terminal I am capturing events that won't come up there
az iot hub monitor-events --hub-name redacted-name --output table

My mistake, please remove the end parameter.
e.g.
str(term.read(15).decode())

@cartertinney Thank you for following up
I tried removing the end tail
but it despite showing no errors it wuldn't show any messages either monitoring them with
az iot hub monitor-events --hub-name redacted-name --output table

@AndreV84
Can you try this instead?
az iot hub monitor-events --hub-name <your IoT Hub name> --output json

--output json
Starting event monitor, use ctrl-c to stop...

it is empty if concurrently running the script, but mosquitto pub messages if I run them would appear there
  "module": "",
        "interface": "",
        "component": "",
        "payload": "{\"key\": \"value\"}"
    }
}

Could you try at your side with the following emulated gps ?
terminal 1:
user@unit:~$  socat -d -d pty,raw,echo=0 pty,raw,echo=0
2021/05/06 13:13:55 socat[23720] N PTY is /dev/pts/25
2021/05/06 13:13:55 socat[23720] N PTY is /dev/pts/26
2021/05/06 13:13:55 socat[23720] N starting data transfer loop with FDs [5,5] and [7,7]

terminal 2:
sudo apt install csh -y
wget https://iweb.dl.sourceforge.net/project/gpsfeed/gpsfeed-latest.zip
unzip gpsfeed-latest.zip
user@unit:~$ tclsh8.6 gpsfeed+.tcl

From this point there will be opened GUI of the gpsfeed. - two GUI windows;
e.g. in serial section goes insteead of default COM1 -> /dev/pts/25; baudrate to ->9600
Then in second GUI window = pressing play [ in the top left - a circle icon with a dot in the center;]
Now from the third terminal the GPS feed is available
:~$ cat /dev/pts/26
$GPGGA,180756,3756.2421,N,02346.8948,E,1,04,5.6,724.4,M,34.5,M,,*43

@AndreV84
Can you provide the details of this GPS emulator? I am not familiar with this particular software you are using or how to set it up. I can't repro your issue without specific instructions.

hi @cartertinney
thank you for following up!
How is it going?
You may find exact instructions in the previous message.
Let me know if it worked or failed at a certain step
Thanks

@AndreV84
I'm trying to follow along, but I run into the error "can't find package Tk" when executing the tclsh8.6 command.

@cartertinney Thank you for following up!
The error seems due to the use of more obsolete version of  tclsh8.3.
did you try the 8.3 version?
Loading Tk as a standard package is not supported in 8.3 (which is basically unsupported now anyway; the last 8.3 release was about 10 years ago). Upgrading to 8.5 should make it all work; it might also work in 8.4 (but that's also end-of-lifed now).

(There's a whole bunch of reasons why it didn't work in 8.3; basically, it required extra API functions to be added – done in 8.4 – and it required the build to be done in the right way. That was done in 8.4 too.

could you try with
sudo apt-get install -y tcl8.6

then
tclsh8.6 gpsfeed+.tcl

Thanks

I am using 8.6

could you try after
 sudo apt-get install tk-dev

sudo apt install tk

Thanks

Now I get
no display name and no $DISPLAY environment variable
    while executing
"load /usr/lib/x86_64-linux-gnu/libtk8.6.so Tk"
    ("package ifneeded Tk 8.6.5" script)
    invoked from within
"package req Tk"
    (file "gpsfeed+.tcl" line 34)

is it a headless execution without display conencted? from bare metal linux computer or from emulated linux inside VM?
The message indicates there is no display variable detected by Linux OS.
What will be the output of executing of the line below? It would return all Displays detected:
cd /tmp/.X11-unix && for x in X*; do echo ":${x#X}"; done ; cd -

The program has GUI that is supposed to pop up as graphical interface to the X11 display.
Typically the display parameter if detected could be exported like
export DISPLAY=:0

then the command execution will return the output to the display conencted, unless its identifier is differnt than 0

I'm not running a display. I don't really need to see a GUI, I really just need to get the GPS emulator to start outputting data, so I can read it in from the Python code. Can you provide me simple steps to simply run the GPS emulator so I can read from it in Python?

@cartertinney this particular emulator has GUI implementation
from GUI it starts,
then it could be accessed from terminal/python as long as it gets running

for terminal [ [ no display ] ]
we could use another tool - gpsfake
sudo apt install python-gps  -y
wget https://raw.githubusercontent.com/esutton/gps-nmea-log-files/master/TrimbleR1_20160310-165531.txt

gpsfake -c 0.1 -P 4000 TrimbleR1_20160310-165531.txt

then they could be opened as socket or read with e.g.
gpsmon localhost:4000

proposed simple steps [ requires display vizualization ] to start the emulator
https://github.com/Azure/azure-iot-sdk-python/issues/771#issuecomment-853237570
once the emulator got started it could be read from python e.g .with
import sys

with open("/dev/pts/26", "wb+", buffering=0) as term:
        while True:
        	print(term.read(26).decode(), end='')
        sys.stdout.flush()

here is another complex though example how to read serial gps given it is present on a serial port
import serial

SERIAL_PORT = "/dev/serial0"
running = True

# In the NMEA message, the position gets transmitted as:
# DDMM.MMMMM, where DD denotes the degrees and MM.MMMMM denotes
# the minutes. However, I want to convert this format to the following:
# DD.MMMM. This method converts a transmitted string to the desired format
def formatDegreesMinutes(coordinates, digits):

    parts = coordinates.split(".")

    if (len(parts) != 2):
        return coordinates

    if (digits > 3 or digits < 2):
        return coordinates

    left = parts[0]
    right = parts[1]
    degrees = str(left[:digits])
    minutes = str(right[:3])

    return degrees + "." + minutes

# This method reads the data from the serial port, the GPS dongle is attached to,
# and then parses the NMEA messages it transmits.
# gps is the serial port, that's used to communicate with the GPS adapter
def getPositionData(gps):
    data = gps.readline()
    message = data[0:6]
    if (message == "$GPRMC"):
        # GPRMC = Recommended minimum specific GPS/Transit data
        # Reading the GPS fix data is an alternative approach that also works
        parts = data.split(",")
        if parts[2] == 'V':
            # V = Warning, most likely, there are no satellites in view...
            print "GPS receiver warning"
        else:
            # Get the position data that was transmitted with the GPRMC message
            # In this example, I'm only interested in the longitude and latitude
            # for other values, that can be read, refer to: http://aprs.gids.nl/nmea/#rmc
            longitude = formatDegreesMinutes(parts[5], 3)
            latitude = formatDegreesMinutes(parts[3], 2)
            print "Your position: lon = " + str(longitude) + ", lat = " + str(latitude)
    else:
        # Handle other NMEA messages and unsupported strings
        pass

print "Application started!"
gps = serial.Serial(SERIAL_PORT, baudrate = 9600, timeout = 0.5)

while running:
    try:
        getPositionData(gps)
    except KeyboardInterrupt:
        running = False
        gps.close()
        print "Application closed!"
    except:
        # You should do some error handling here...

from  https://drive.google.com/file/d/1TODVFoy82rdhfwawN9vUhAGCsYRyk-7V/view

example:
terminal 1
 socat -d -d pty,raw,echo=0 pty,raw,echo=0
2021/06/07 20:13:28 socat[19529] N PTY is /dev/pts/17
2021/06/07 20:13:28 socat[19529] N PTY is /dev/pts/18
2021/06/07 20:13:28 socat[19529] N starting data transfer loop with FDs [5,5] and [7,7]

terminal 2
 tclsh8.6 gpsfeed+.tcl

terminal 3:
python-gps-examples$ python serial_gps.py
Application started!
Your position: lon = 024.005, lat = 38.314
Your position: lon = 024.745, lat = 38.108
Your position: lon = 024.993, lat = 38.993
Your position: lon = 024.251, lat = 38.083

@AndreV84
I do not have a display, I am running a linux VM trying to repro this issue. The non-GUI option you have provided also doesn't seem to be working for me.
Is there a way for me to repro your issue without a display? I really just need output that I can read from in GPS serial format to see why you are having problems sending messages - how the GPS itself works is irrelevant, the only important thing is to get a data stream that resembles a GPS so I can reproduce the issue in sending messages.

@cartertinney
maybe you could run X11 session on Linux VM? it should work to run the GUI
We did not test with other emulators, but gpsfeed+ with consistent confirmed results reading from python

However, you may try with just
socat -d -d pty,raw,echo=0 pty,raw,echo=0
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/2
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/3

it will make two pts tty devices;
try directing any text output to the /dev/pts/2
echo "Test" > /dev/pts/2

then try reading it from third terminal
cat < /dev/pts/2

as long as
you can see the message it would be good to try with python;
No GUI required
source https://stackoverflow.com/questions/52187/virtual-serial-port-for-linux

@AndreV84
I have been able to successfully get data being sent over messages while emulating GPS data using the following code:
from azure.iot.device import IoTHubDeviceClient, Message

conn_str = "<your connection_string_here>"
client = IoTHubDeviceClient.create_from_connection_string(conn_str)
client.connect()

pty = "/dev/pts/4"    # change this to whatever pty entry you are receiving on

with open(pty, "wb+", buffering=0) as term:
    while True:
        data = term.readline()
        message = Message(data)
        client.send_message(message)

@cartertinney
Thank you very much!
It seems a first true positive implementing proof of concept transmission of GPS data that I can repeat at my device;
However, I shall further investigate how to send separate set of values for values to be delivered to Azure Iot as separate
longitude, lattitude values versus time.

@AndreV84
Based on the data output I saw from your emulator above, I would probably do something like
data = term.readline()
values = data.split(",") # this splits the data on comma, and since your data is comma-separated, it should work for you
for value in values:
     msg = Message(value)
     client.send_message(msg)

Not completely sure that'll work for your purposes though. Either way, that's a question for you as to how you want to split up the data. It has nothing to do with the IoTHub Client. As you can see, the Client doesn't care what you send in - just create a Message with the data, and send.
If this has answered all your concerns, can I go ahead and close this issue now?

@cartertinney
Thank you for following up!
I shall check with my team and will get back to you regarding the issue.
Thank you very much!

@AndreV84
Yes. Just do what was in that sample to parse your GPS output like in the getPositionData function. Take that logic, and then put your desired output into messages, and send them using a Message to IoTHub.
I want to reiterate that Azure IoT has nothing to do with your GPS. You format your data however you want from whatever source you want. Then you provide that string data to the IoTHubClient, and it sends it. It does not matter from the client's perspective what the data is.

@cartertinney
given we have a GO code that sends GPS data  to local mqtt server[ if to specify local mqtt server as target]
can we send GPS data from the very same GO code by specifying Azure IoT server name instead? How do we add authentication towards Iot for GO code? Is it something that can be supported ?

we will try to test with bare metal GPS unit
if it works the issue we can close
thank you very much

@AndreV84
Authentication is difficult, this is the main reason to use an IoTHubClient or our other Python tooling. Unfortunately, we do not support Golang at this time - only Python, NodeJS, C#, Java and C.
If you wish to try and authenticate manually, this document describes how to do it, but doesn't specifically describe how to do so with Golang, however the general principles described in here still apply.
Furthermore, unless you have a remaining issue directly with using IoTHubClients in the Python SDK, I will have to close this issue, as both Golang and manual connection bypassing the IoTHubClient is outside of my scope as a developer on the Python SDK.

@cartertinney
Thank you for following up!
Yes we tried this generic instruction document to use SAS to send to Azure IoT from
mosquitto_pub
mosquitto_sub
it would work for simple messages;
However we planned to test bare metal GPS unit with python cli SDK next days, I will update if it did not work; There might be various reasons why it still might fail with sdk, e.g. due to some configutations or permissions

@AndreV84
I'm going to close this issue, as the original issue has been addressed. If you run into further problems with the IotHub clients, please open a new issue.

@cartertinney Hi, How is it going?
Thank you for your help.
Could you extend if we could connect ZeroMQ with Azure Iot, please?

@AndreV84
In what context? The Azure IoT Device Client uses Paho for it's messaging to the IoT Hub and that is not configurable. If you want to receive data from another source using ZeroMQ and then send that data to the Hub, that should work fine, as again, the package for Device Clients does not care where your data comes from or what your application looks like.
Also, as I mentioned before, please open a new issue if you have further concerns - we do not regularly check closed issues for follow-ups

@cartertinney
Thank you for following up.
We are still trying to integrate video+gps from nvidia jetson devices to Azure IoT infrastructure.
As in https://github.com/Azure-Samples/NVIDIA-Deepstream-Azure-IoT-Edge-on-a-NVIDIA-Jetson-Nano/issues/19#issuecomment-881207125

I can't say I have a particular insight into that scenario, but as I said, if you wish to send data to Azure IoT Hub, the Device Client will do it over MQTT. The client does not support any other protocols for sending data to Azure IoT Hub.


Chrome debugger extension - debugger always stops in main.js for Angular 10 project

If this is not the right forum I apologize. I could not find a chrome debugger forum that was open to posts.
Using "ng serve" to serve my project code on port 4200, I launch my Chrome debugger using Run - Start Debugging in VS Code 1.54.3.
Other software:
Chrome 87.0.4280.141.
Node.js 12.18.3.
Windows 10 (10.0.17134).
Electron 11.3.0.
V8: <IP_ADDRESS>-electron.0
The code always stops on line main.js:4002 which contains this code only:
/**/ }),
There is no breakpoint set on this line. There are no breakpoints set in main.ts. There are no breakpoints listed in the Run/Debug sidebar. There are no breakpoints set in Chrome's F12 console.
I've tried editing my main.ts file and adding / removing carriage returns just to "touch" the file. This seemed to get the code to execute one time without hitting the breakpoint but I was unable to reproduce this behavior. Prior to a few days ago, this was not happening. However, many months ago it did happen. I do not recall what made it stop.
If anyone knows why this happens and if anything can be done to reset it short of something drastic like reinstalling VS Code, Chrome, or heaven-forbid Windows, I'd appreciate learning about it. This is super annoying when developing and debugging a sizable application. Thanks a bunch.

/jsDebugLogs

After examining the trace log, there is a lot of personal information in there, namely urls with querystring params for my local organization. Do you have any suggestions for scrubbing these files before uploading? I do not want to be the guy who uploaded sensitive information and ends up on the 5 o'clock news :)

You can unzip the file to text, and then remove or redact the sensitive parts. Alternately, if you can create a small sample repo where this problem happens, that would also work well 🙂

I think it would be impossible to repro since it involves some sort of unique circumstance under which it happens. But I will eliminate any personal info and re-zip. Thanks for the fast response!

I attached the trace file.
To restate what I am experiencing, during the launch of the Angular application, the code breaks on line 4010 of main.js. This is the last line of a section that looks like this:
"use strict";
__webpack_require__.r(__webpack_exports__);
/* harmony default export */ __webpack_exports__["default"] = ("<div class=.........
/***/ }),   <-- this is line 4010

Other info:
Line 4008 is enormous. It is so long that VS Code will not even render the end of the line if I put my cursor on it and press END. The cursor just disappears somewhere.
When I press F5 to allow the app to continue to load, then clear the debug console and re-load the browser page, these are the only three debug lines that are printed before the 4008/4010 breakpoint is hit:
Could not read source map for chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/commons.js: Unexpected 503 response from chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/commons.js.map: Unsupported protocol "chrome-extension:"
Could not read source map for chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/timestampManager.js: Unexpected 503 response from chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/timestampManager.js.map: Unsupported protocol "chrome-extension:"
Could not read source map for chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/documentStartIndex.js: Unexpected 503 response from chrome-extension://olpofapmbejnopkdjkdejhomgkbbdojd/documentStartIndex.js.map: Unsupported protocol "chrome-extension:"

vscode-debugadapter-b30f9523.gz

Thanks for the log. In there I'm not seeing an "stopped" events which is what is sent to signal to VS Code that it's paused on a line. Did those get removed? Without them it's hard to see where the issue is happening.

I don't think so unless those same lines would have a url on them. What is the search string I can look for? I will look at the original log text.
Also, the extension that is throwing the 503 errors is something called WalkMe. It apparently lets IT support monitor or overlay information onto the browser window. I'd like to remove it but I don't think I can. It's odd that it was just added to my computer in the last couple of weeks. This sort of corresponds to the time I started fighting this break issue.

Maybe this will help. While the code was frozen at the "breakpoint", I decoded the trace file. These are the last four lines while VSCode is waiting on me to press F5 (continue):
{"tag":"cdp.receive","timestamp":1616789869475,"metadata":{"connectionId":0,"message":{"id":1368,"result":{"result":[]},"sessionId":"7E51AB92BE538C9A20C32396EE200EFA"}},"level":0}
{"tag":"cdp.receive","timestamp":1616789869476,"metadata":{"connectionId":0,"message":{"id":1369,"result":{"result":[{"name":"module","value":{"type":"object","className":"Object","description":"Object","objectId":"5225363507310389409.2.52","preview":{"type":"object","description":"Object","overflow":false,"properties":[{"name":"i","type":"string","value":"C9SC"},{"name":"l","type":"boolean","value":"false"},{"name":"exports","type":"object","value":"Module"}]}},"writable":true,"configurable":true,"enumerable":true,"isOwn":true},{"name":"__webpack_exports__","value":{"type":"object","className":"Module","description":"Module","objectId":"5225363507310389409.2.53","preview":{"type":"object","description":"Module","overflow":false,"properties":[{"name":"default","type":"string","value":"
\n    </confirmation-dialog>\n
"},{"name":"__esModule","type":"boolean","value":"true"},{"name":"Symbol(Symbol.toStringTag)","type":"string","value":"Module"}]}},"writable":true,"configurable":true,"enumerable":true,"isOwn":true},{"name":"__webpack_require__","value":{"type":"function","className":"Function","description":"function __webpack_require__(moduleId) {\n/******/\n/******/ \t\t// Check if module is in cache\n/******/ \t\tif(installedModules[moduleId]) {\n/******/ \t\t\treturn installedModules[moduleId].exports;\n/******/ \t\t}\n/******/ \t\t// Create a new module (and put it into the cache)\n/******/ \t\tvar module = installedModules[moduleId] = {\n/******/ \t\t\ti: moduleId,\n/******/ \t\t\tl: false,\n/******/ \t\t\texports: {}\n/******/ \t\t};\n/******/\n/******/ \t\t// Execute the module function\n/******/ \t\tmodules[moduleId].call(module.exports, module, module.exports, __webpack_require__);\n/******/\n/******/ \t\t// Flag the module as loaded\n/******/ \t\tmodule.l = true;\n/******/\n/******/ \t\t// Return the exports of the module\n/******/ \t\treturn module.exports;\n/******/ \t}","objectId":"5225363507310389409.2.54"},"writable":true,"configurable":true,"enumerable":true,"isOwn":true}]},"sessionId":"7E51AB92BE538C9A20C32396EE200EFA"}},"level":0}
{"tag":"dap.send","timestamp":1616789869479,"metadata":{"connectionId":1,"message":{"seq":1927,"type":"response","request_seq":31,"command":"variables","success":true,"body":{"variables":[{"name":"module","value":"{i: 'C9SC', l: false, exports: Module}","evaluateName":"module","type":"object","variablesReference":3},{"name":"__webpack_exports__","value":"Module {default: '
\\n   …irmed($event)\"></confirmation-dialog>\\n
', __esModule: true, Symbol(Symbol.toStringTag): 'Module'}","evaluateName":"__webpack_exports__","type":"object","variablesReference":4},{"name":"__webpack_require__","value":"ƒ __webpack_require__(moduleId) {\\n/******/\\n/******/ \\t\\t// Check if module is in cache\\n/******/ \\t\\tif(installedModules[moduleId]) {\\n/******/ \\t\\t\\treturn installedModules[moduleId].exports;\\n/******/ \\t\\t}\\n/******/ \\t\\t// Create a new module (and put it into the cache)\\n/******/ \\t\\tvar module = installedModules[moduleId] = {\\n/******/ \\t\\t\\ti: moduleId,\\n/******/ \\t\\t\\tl: false,\\n/******/ \\t\\t\\texports: {}\\n/******/ \\t\\t};\\n/******/\\n/******/ \\t\\t// Execute the module function\\n/******/ \\t\\tmodules[moduleId].call(module.exports, module, module.exports, __webpack_require__);\\n/******/\\n/******/ \\t\\t// Flag the module as loaded\\n/******/ \\t\\tmodule.l = true;\\n/******/\\n/******/ \\t\\t// Return the exports of the module\\n/******/ \\t\\treturn module.exports;\\n/******/ \\t}","evaluateName":"__webpack_require__","type":"function","variablesReference":5},{"name":"this","value":"Module","evaluateName":"this","type":"object","variablesReference":6},{"name":"Return value","value":"undefined","evaluateName":"Return value","type":"undefined","variablesReference":0}]}}},"level":0}
{"tag":"dap.send","timestamp":1616789872308,"metadata":{"connectionId":1,"message":{"seq":1928,"type":"event","event":"output","body":{"category":"telemetry","output":"js-debug/dap/operation","data":{"errors":[],"breakpointLocations":{"operation":"breakpointLocations","totalTime":2.9,"max":2.9,"avg":2.9,"stddev":null,"count":1,"failed":0},"!breakpointLocations.errors":[],"threads":{"operation":"threads","totalTime":0.7,"max":0.7,"avg":0.7,"stddev":null,"count":1,"failed":0},"!threads.errors":[],"stackTrace":{"operation":"stackTrace","totalTime":16.8,"max":15.7,"avg":8.4,"stddev":10.3,"count":2,"failed":0},"!stackTrace.errors":[],"source":{"operation":"source","totalTime":95.6,"max":95.6,"avg":95.6,"stddev":null,"count":1,"failed":0},"!source.errors":[],"scopes":{"operation":"scopes","totalTime":25.5,"max":25.5,"avg":25.5,"stddev":null,"count":1,"failed":0},"!scopes.errors":[],"variables":{"operation":"variables","totalTime":7.6,"max":7.6,"avg":7.6,"stddev":null,"count":1,"failed":0},"!variables.errors":[]}}}},"level":0}

Okay, I figured out what was causing it. One of the breakpoints in my list on the sidebar was considered "unbound" even though when I clicked on it, it took me to one of my Angular html templates (line 22 or whatever). When I disabled this unbound breakpoint, the app ran fine without stopping. When I re-enabled it, it stopped the app in main.js. So it appears that the VSCode debugger will stop when there's an unbound breakpoint. It also appears this is not being reflected in the trace.

It's hard for me to tell what's happening with the limited info/logs.
It is by design that the debugger pauses when evaluating scripts with sourcemaps, but it should then unpause after parsing the sourcemap. However stopping in an html template would indicate to me that something else is also problematic with your sourcemaps, which _could cause us to remain paused (if a breakpoint evaluates to the currently-paused position).

Interesting. Since the "ng serve" command is responsible for doing a debug compile and presumable providing any source maps for a template / component, I'm not sure what could be happening there. Those are some deep weeds I've never wander into :)
For testing purposes, it may be possible to create an "unbound breakpoint" (maybe by putting a breakpoint on a line of an html template and then replacing that line with, say, just a carriage return) and recreate a condition that causes VSCode to break and fail to resume. If so, I suppose the VSCode team would have to decide what to do about that scenario.
Anyway, I'll try it and see what happens.

Sure enough, it seems easy enough to reproduce the scenario. Stackblitz usually takes me forever to set up an example so, while I try to do that, here's all I think you'd need to see this happen.
Make an Angular 2+ component that is not involved in the startup of the app or on the main page in any way. In the HTML, put these three lines and set a breakpoint on the second one:

  {{myVariable}}  <-- set breakpoint on this line

After saving, start "ng serve". Start the VSCode debugger which should try to load the app. If you have a breakpoint listed as "unbound" for this component (clearly it should not be considered unbound since it is in the template), you should see VSCode stop in main.js before it even tries to display anything.


Same endpoint in different schemas not working

Scenario: You have the same endpoint different schemas, but with different http methods. E.g. schema1 has GET /books and schema2 has POST /books. Ideally there should be no problem in merging such two schemas. Right now either GET /books or POST /books will be left out of the result.

@klyver
Could you please provide more details
What version of gradle?
what version of the plugin?
Sample input yaml files

Sure.
gradle version 6.6.1
openapi-merger-gradle-plugin version 1.0.4.
bookget.yaml:
info:
  title: OpenAPI definition
  version: v0
servers:
  - url: http://localhost:8080
    description: Generated server url
paths:
  /api/book/{name}:
    get:
      tags:
        - book-controller
      operationId: getBook
      parameters:
        - name: name
          in: path
          required: true
          schema:
            type: string
      responses:
        "200":
          description: OK
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/Book'
components:
  schemas:
    Book:
      type: object
      properties:
        name:
          type: string
        iban:
          type: string

and bookpost.yaml
info:
  title: OpenAPI definition
  version: v0
servers:
  - url: http://localhost:8080
    description: Generated server url
paths:
  /api/book/{name}:
    post:
      tags:
        - book-controller
      operationId: getBook
      parameters:
        - name: name
          in: path
          required: true
          schema:
            type: string
      requestBody:
        description: The state altering event
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Book'
      responses:
        "200":
          description: OK
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/Book'
components:
  schemas:
    Book:
      type: object
      properties:
        name:
          type: string
        iban:
          type: string

@klyver
This is a defect and can be addressed in the next version.

That sounds great!

Are there any news regarding this topic?

Will test and then do a release as soon as possible

Released in 1.0.5


Add Timezone Selection on First Login Screen

Modify the first-login screen to include a timezone field for users to set their timezone.

Autofill the timezone field based on the user's browser timezone to simplify the setup process.

We decided that would be better get the timezone from user machine to show the values.


Check Firefox compatibility for Windows

As it looks like there might be some incompatibilities with our CDP implementation when Fission is enabled on Windows. This was mentioned in #7659.
I'm going to force disable Fission for now. Maybe this helps to make the failure go away.

Forcing Fission disabled doesn't actually fix the test. So some other change that I might have done recently in Firefox might have caused it.
I'll check with custom Firefox builds where it might have been introduced.

The problem that we currently have with Windows is that Firefox uses a launcher process that initially gets started, and then itself launches the real Firefox process. As such Puppeteer only knows about the launcher process, and that is the process that gets killed by a couple of tests. By doing that the real Firefox process remains running and is blocking the temporary profile from being removed. And these are the failures that happen most right now.
As such I would suggest to find a solution for that problem first, which means that we would need to fix https://bugzilla.mozilla.org/show_bug.cgi?id=1656962 on our side. Requiring client to find the real process I don't see as a possible solution.
CC @OrKoN.

@whimboo thanks for the info, can we disable that test for FireFox on Windows for now?

@whimboo thanks for the info, can we disable that test for FireFox on Windows for now?

It might not be only a single test. Basically each test that causes BrowserRunner.kill() to be executed.
But hey, I just saw some code which looks kinda weird! Why are we trying to remove the temporary user data directory BEFORE we kill the browser process?
Is that somehow on purpose? From our point of view this will clearly cause issues when the process still has hold on some of the files of the profile. Can we flip that around? I think that this might already help in some cases.

I created https://github.com/puppeteer/puppeteer/pull/7761 to check that.

@OrKoN we also kill the browser here: https://github.com/puppeteer/puppeteer/blob/790c7a0eb92291efebaa37e80c72f5cb5f46bbdb/src/node/Launcher.ts#L374-L396.
So what I see is that when waitForTarget() fails for the initial page, we also kill the browser while at this time we could also call browser.close() for a soft-close. Any reason why this isn't done?

We landed a fix in Firefox which connects the launcher process as used by default on Windows with all the other Firefox processes within the same job. At the same time we also set --remote-debugging-port to automatically use the --wait-for-browser command line argument on Windows, which is required by automation clients to wait for the real Firefox process to be launched. With these two changes the results on Windows are looking way better now!
Nevertheless there are remaining issues, and maybe I can continue next week.

Actually the test Launcher.spec should filter out ignored default arguments causes troubles. In case of Firefox it removes the --no-remote and --headless command line arguments. This means the following:

Without --no-remote specified an existing Firefox process will be re-used, and a new Firefox with the Puppeteer profile will not be spawned.
Removing --headless will launch the browser in non-headless mode and ignoring the HEADLESS setting.

While this might work for Chrome it's not appropriate for Firefox. Also depending on the platform and some other options different default arguments are getting set for Firefox, so the above problem will not only happen on Windows.
@OrKoN any suggestions? IMHO we could update the test so that it only removes the very last argument, which seems to always be the about:blank one?

@whimboo sorry, I am not really familiar with this part. It looks like ignoreDefaultArgs is an optional parameter (https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions). Do you mean that the test is failing on FF? I didn't get the suggestion about updating the test to only remove the last argument.

@whimboo sorry, I am not really familiar with this part. It looks like ignoreDefaultArgs is an optional parameter (https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions). Do you mean that the test is failing on FF? I didn't get the suggestion about updating the test to only remove the last argument.

Ok, so IMHO there should be a distinction between default arguments and mandatory arguments. As such it might be that some of the currently used default arguments for Firefox should actually be mandatory:
https://github.com/puppeteer/puppeteer/blob/4c3caaa3f99f0c31333a749ec50f56180507a374/src/node/Launcher.ts#L447
First the --no-remote argument. It's used across platforms to always create a new instance of Firefox. If that argument is removed (as what the test does) a new window for an already existing Firefox process will be opened. As such this causes side-effects for any following test.
Beside that the test also removes the 3rd default argument which in case of Windows and a headless CI job (which is done by default) will be --headless.
As such I would propose the following:

We move the --no-remote argument to launch(), which would be [similar to --remote-debugging-port] argument, and does NOT allow to ignore it.

We update the test so that it removes one or two other default arguments that are really optional. Not sure if we could even push some other non existent arguments as defaults first, to get these removed before starting the browser.

@whimboo is there a case when a user might want to remove --no-remote so that a new window for an already existing Firefox process will be opened?  In general, I think it makes sense to move the flags that are mandatory out of the default arguments. Thanks for the explanation! Also, does it make sense to have separate tests for Firefox and Chromium to make sure that the right arguments are removed/not removed?

@whimboo is there a case when a user might want to remove --no-remote so that a new window for an already existing Firefox process will be opened? In general, I think it makes sense to move the flags that are mandatory out of the default arguments. Thanks for the explanation! Also, does it make sense to have separate tests for Firefox and Chromium to make sure that the right arguments are removed/not removed?

FYI a patch for that actually landed in #8233 and #8250. Now there are two distinct tests for both Chrome and Firefox. I'll rebase and trigger new checks in a moment.

Note that right now there seems to be only a single remaining failure on Windows which would prevent us from re-enabling results in CI. The failing test is:

 Page.evaluate
   should transfer 100Mb of data from page to node.js:

 Page.evaluate
   should transfer 100Mb of data from page to node.js:

@OrKoN the error trace as visible in the CI logs is far from being helpful. Do you see what could be wrong here? I'm not able to see what is actually failing here, and I cannot reproduce it locally on my local Windows machine.

@whimboo I am not sure what exactly fails but if I read the code right: a CDPSession receives a message that has an ID (thus, a callback is expected as the message is a response to a message from Puppeteer); yet the callback is missing so it looks like an unsolicited response from the browser. I suggest adding DEBUG=puppeteer.* env var to the workflow and inspect the CDP traffic.

Ah sorry, you should also probably include a .only (possible with a eslint comment disabling rules we might have). Otherwise, it seems that logging would cause jobs to time out.

As it looks like the job on Windows doesn't finish at all. It's running 5:30h now and I think that I should cancel. Not sure how to continue investigation given that I cannot reproduce locally.
https://github.com/puppeteer/puppeteer/runs/6171558887?check_suite_focus=true

I think it actually might be the problem with GitHub actions. Let's try to restart it and see if it times out again. I can also try on my Windows locally.

Unfortunately, I am unable to repro on my Win too :(

Let me run the full test suite: perhaps there is some state leaking from previous tests.

I have the following failures locally with this PR (not sure if I have any failures on main).

Hm, this is sad to hear that you cannot reproduce the failure locally and that the CI checks do not have the failures that you are seeing locally. :(
As such I think that due to time constraints I'll have to post-pone any work on this PR for some time.

I believe we can close this one.

@OrKoN, are tests on Windows run again for Firefox? I cannot remember if / that we enabled those.

@whimboo no, we have not enabled them


Copy .asf.yaml to asf-site branch

I think the reason the deploy didn't work was because the .asf.yaml file has to be on the asf-site branch.  This PR changes that.  It also tweaks the orphan-deploy process a little bit because the old one was leaving behind cached files.  Now it removes all files, downloads the book, then uploads everything.

Awesome, LGTM!

Looks like this did the trick: https://arrow.apache.org/cookbook/ is now live. :tada:

Cool, thanks!


[$500]  LHN - Chat disappear from LHN when user haven't sent any message but create Private Note

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Navigate to staging.new.expensify.com
Click on FAB > Send Message
Select User B you never have chat before
Don't send any message
Create Private Note with this User
Navigate to other chat

Expected Result:
Since User A create Private Note with other User B the conversation should remain in LHN
Actual Result:
Chat disappear from LHN when user haven't sent any message but create Private Note
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

Version Number: 1.3.75-0
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/c8450d78-c8ec-4ad6-9ffd-f49c89233477
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01e5a66919b6dfefd2
Upwork Job ID:<PHONE_NUMBER>309785600
Last Price Increase: 2023-10-02

@burczu can you please review the proposal above? Thanks!

@allroundexperts Thanks for detailed proposal. I've just reviewed and tested it and I think it does identify the issue correctly. Also the provided solution seem to be accurate, so I think we can proceed with it.
🎀 👀 🎀 C+ reviewed

Great proposal, let's move forward with that fix!

@robertjchen @isabelastisser Plase take a look at the comment @allroundexperts posted in his PR: https://github.com/Expensify/App/pull/29096#issuecomment-1753273444
It looks like we need to put this PR on hold, and switch to Internal for now to first update the backend. What do you think?

 Do we ned to put this on hold? thanks!

Yes, I would agree- let's put it on Hold for now while we look into the Internal aspect.

Still on hold for now

Hey @isabelastisser! I'm leaving C+ role today and joining Waves. Please assign another C+ instead of me. Thank you!


Integrating Doce with Jean-zay

For now, I launch jobs on Jean-zay using sbatch script.slurm with script.slurm containing the code found here : http://www.idris.fr/jean-zay/gpu/jean-zay-gpu-exec_mono_batch.html
The last line of this script is running my training python code, with parameters, such as : python -u train.py --name exp_name --steps 1200 --batch 16
To make use of Doce, a specific .slurm script should be run for a set of specific settings, which corresponds to one particular job. So in order to run the training for multiple settings, Doce should create one .slurm file for each set of settings, in which the last line runs the main Doce script for only one setting : python main.py -c -s setting1=...+setting2=...
Note that other lines of the .slurm script might be custom for a set of setting, e.g., #SBATCH --output=gpu_mono%j.out and #SBATCH --error=gpu_mono%j.out which contain the outputs and errors of the job. Could be interesting to name them after the corresponding set of settings.
These output files contain both Python and runtime errors, so you might want to adapt Doce error catching to know if the job has been finished without any error or not.
Further works: it could be nice to add a routine to split a training in multiple jobs if the total script time is more than 20 hours (Jean-zay limit). I follow this way of doing sequential jobs : http://www.idris.fr/jean-zay/cpu/jean-zay-cpu-exec_cascade.html

the new version has a -j filename.
The filename should be a slurm template. Example for jean zay attached
jz.slurm.txt

small doc available using -h


Added arm ids list

Added arm ids list to the initialize method, which enables to identify arms by given ids instead of ordered index number.

i think the 'last_selected_arms'(l7) is not necessary.

OK!!


Discriminator field is being duplicated

When using an openapi spec with a discriminator similar to this
 Model:
      type: object
      discriminator:
        propertyName: modelType
      properties:
        modelType:
          type: String
      required:
        - modelType

The following class is generated
@javax.annotation.processing.Generated(value = "org.openapitools.codegen.languages.MicronautCodegen", date = "2020-09-07T18:32:03.031189-03:00")
@com.fasterxml.jackson.annotation.JsonTypeInfo(use = com.fasterxml.jackson.annotation.JsonTypeInfo.Id.NAME, include = com.fasterxml.jackson.annotation.JsonTypeInfo.As.PROPERTY, property = "modelType", visible = true)
@com.fasterxml.jackson.annotation.JsonSubTypes({
.....
})
public abstract class Model {

	public static final String JSON_PROPERTY_NODE_TYPE = "modelType";

	@com.fasterxml.jackson.annotation.JsonProperty(JSON_PROPERTY_NODE_TYPE)
	@com.fasterxml.jackson.annotation.JsonInclude(com.fasterxml.jackson.annotation.JsonInclude.Include.ALWAYS)
	private String modelType = null;
....

This generates duplicated entries in the json
{
    "modelType": "TYPE_A",
    "modelType": "TYPE_A",
...

even when the property is set to null:
{
    "modelType": "TYPE_A",
    "modelType": null,
...

Expected behavior: this field is not duplicated.
Ideas:
Either include = com.fasterxml.jackson.annotation.JsonTypeInfo.As.PROPERTY could become  include = com.fasterxml.jackson.annotation.JsonTypeInfo.As.EXISTING_PROPERTY or  com.fasterxml.jackson.annotation.JsonInclude.Include.ALWAYS could be replaced by com.fasterxml.jackson.annotation.JsonInclude.Include.NOT_NULL.
The first proposal would force the user to set this value by himself. The second would force the user to never set this value.

I prefer proposal #2. I'll implement a fix.

@stephanschnabel Another idea (my preferred one) is to mimic what happens if you define a discriminator but not a property with the same name, like this
MyModel:
      type: object
      required:
        - type
      discriminator:
        propertyName: type
      properties:
        id:
          type: string

The generated model is
@com.fasterxml.jackson.annotation.JsonTypeInfo(use = com.fasterxml.jackson.annotation.JsonTypeInfo.Id.NAME, include = com.fasterxml.jackson.annotation.JsonTypeInfo.As.PROPERTY, property = "type", visible = true)
@com.fasterxml.jackson.annotation.JsonSubTypes({
	.....
})
@io.micronaut.core.annotation.Introspected
public abstract class MyModel {

	public static final String JSON_PROPERTY_ID = "id";

	@com.fasterxml.jackson.annotation.JsonProperty(JSON_PROPERTY_ID)
	@com.fasterxml.jackson.annotation.JsonInclude(com.fasterxml.jackson.annotation.JsonInclude.Include.NON_NULL)
	private java.lang.String id;
.... // no reference to type

I've create a pr:

discriminator property is not generated (only via jackson type mapping)
abstract class has abstract getter for discriminator
every subtype implements getter with his discriminator
test for string / enum discriminator


change version constraint before next release

'yiisoft/yii2' version at composer.json should be changed to ">=2.0.5" before next release.

Resolved by commit 8bcfdee0040ead2c399fd325ffad4bb6299c7fc7


Include module nf_conntrack in WSL2 kernel

Is your feature request related to a problem? Please describe.
Yes, Istio issue 37885 WSL2 kernel (<IP_ADDRESS>-microsoft-standard-WSL2) does not have nf_conntrack module
Describe the solution you'd like
Include module nf_conntrack
Describe alternatives you've considered
N/A
Additional context
Without the module istio dns proxy will not run. for more details check Istio issue 37885

Thanks @elsaco
How to enable "# CONFIG_NF_CONNTRACK_ZONES is not set"?
Is it by changing in /proc/config.gz and reboot WSL?

@c0ffeec0der to enable CONFIG_NF_CONNTRACK_ZONES download the latest WSL2-Linux_kernel from https://github.com/microsoft/WSL2-Linux-Kernel/archive/refs/tags/linux-msft-wsl-<IP_ADDRESS>.tar.gz and compile a new kernel. After a successful build, copy the new bzImage to the Windows filesystem and reference the new kernel in your .wslconfig file. Run wsl --shutdown to make sure the new kernel is used for new WSL instances.
This is a make config screenshot with the above setting enabled:

@elsaco Thank you very much! I am able to compile the kernel


[Feature Request] option to disable animations

Hello,
Gitea has almost everywhere animations: the "Create" menu, the issue preview...
The problem is, I don't like these animations, I think they're a bit disturbing. So, I think it would be nice if there's an option in the settings to disable these animations.
Thanks!
Can you reproduce the bug at https://try.gitea.io:

[x] Yes
[ ] No

I generally find the animations obnoxious too. Here is a very crude method of doing it, certainly not something we should commit like that:
* {
  animation-duration: 0ms !important;
}

I, personally, like animations.  But perhaps support could be added for the OS preference, which browsers expose in CSS as prefers-reduced-motion.  It can also be accessed in JS.

Seems like this is controlled by the package 'fomatic-ui' used by gitea. I have reported the issue upstream


refactor(sdk/go): use any instead empty interface

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Could, you, please provide some motivation behind this change.

@verytable empty interface says nothing. For this case in Go1.18 have any type. It's only cosmetic change. And code with any looks cleaner than interface{}

I'm just advocating the cleanliness of the description.


CB-18507 - Prepared Images for upgrade.

Adding boolean in upgradeV4Response to track what images are prepared for upgrade.
Added image id to message saved to StackStatus when CLUSTER_UPGRADE_PREPARATION_FINISHED status is saved.
Added boolean field to ImageInfoV4Response to indicate if an image id is already prepared.
Added Unit Tests to validate.

as we agreed on it, please store the prepared image related info in clustercomponent. don't use statusreason for anything like this

done! can you please review again? @lacikaaa

I don't see any cleanup regarding prepared images. After the upgrade is done and the instances are replaced I think the prepared images should be cleaned

@lacikaaa addressed your comments

@vivprabu could you please rebase your branch to top of the master so the swagger check will also pass ?

@vivprabu could you please rebase your branch to top of the master so the swagger check will also pass ?

rebased


Direct integration between platform and learning materials

This features will give the possibility for users to interact with Haaukins from a link provided in the learning materials page. The user should be able to connect to Guacamole and solve the challenges related to the learning material page in which was the link.
The Idea is to create an API in Haaukins that accept as input the challenges tag and create an instance of Guacamole in which the challenges are running. A loading page will be showed to the user while an instance is created and ones is done the user will be redirected to Guacamole. Moreover Haaukins should expose a second api used to solve challenges
A form will be created below the link provided in the learning materials in order to send requests to the API to check if the flags are correct.

@mrturkmen06 Any suggestions are more than welcome :smile:

It is separated from haaukins repository, now it has it is own repo called haaukins-api.


Sound effects support

There should be a way to experience sound in the best way possible. If there is a sound coming out from any particular poly asset the user should have that same experience (and not the experience where the sound comes out from the Android smartphone because that would spoil the UX)
For example) If there are three different virtual objects in a room and they all emit sound simultaneously then the user should experience the loudest sound coming from the object that he or she is standing closer to. However, if the sound comes out from the smartphone the sound will always remain constant i.e irrespective of which object is close to the user and therefore spoiling the UX.
Is there any pre-existing support already for this situation?

We haven't integrated audio with Sceneform, but I think there are libraries that will do what you are looking for.  One library to check out would be Resonance Audio