Error when using matrix in batch mode

Hi,
I use Matrix tool (ORS Tools v1.5.2 with QGIS 3.16.4 or 3.22.4) in batch mode.
When I enter parameters manually, only the first process fails. But, the same process works fine if I execute in a stand alone operation. When I upload a json file, all processes fail. Parameters are identical, except INPUT_START_LAYER and INPUT_END_LAYER (layer's name is automatically replaced by the database connection string in json file).
Last week, I successfully do the same task with the same tables and the same parameters. So, I don't understand...
Output files are created but some rows are missing.
Thank you for your help
I get this message when entering the parameters manually:

Algorithme de traitement 1/31…
Démarrage de l'algorithme Matrix from layers…
Paramètres en entrée:
{'INPUT_AVOID_BORDERS': None,
'INPUT_AVOID_COUNTRIES': '',
'INPUT_AVOID_FEATURES': [],
'INPUT_AVOID_POLYGONS': None,
'INPUT_END_FIELD': 'esz',
'INPUT_END_LAYER': 'zec_id_1_a_3499_v_eb4b4925_ef3b_496a_80c7_8246bbe43602',
'INPUT_PROFILE': 0,
'INPUT_PROVIDER': 0,
'INPUT_START_FIELD': 'code',
'INPUT_START_LAYER': 'cis_als_v_33dfe97a_e7d4_424f_bd08_49bc1709fdca',
'OUTPUT': <QgsProcessingOutputLayerDefinition {'sink':L:/PREVISION/SIG/bd_opensis/ro/matrices/vl_h_1_cis_als_v.csv, 'createOptions': {}}>}
Traceback (most recent call last):
File "C:/Users/v-n/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\ORStools\proc\matrix_proc.py", line 189, in processAlgorithm
duration = response['durations'][s][d]
IndexError: list index out of range
L'algorithme Matrix from layers a échoué…
Execution failed after 2.75 seconds

I get this message when I upload json file (I just deleted DB connection parameters):

Algorithme de traitement 1/31…
Démarrage de l'algorithme Matrix from layers…
Paramètres en entrée:
{'INPUT_AVOID_BORDERS': None,
'INPUT_AVOID_COUNTRIES': '',
'INPUT_AVOID_FEATURES': [],
'INPUT_AVOID_POLYGONS': None,
'INPUT_END_FIELD': 'esz',
'INPUT_END_LAYER': "postgres://dbname='lizmap_opensis_sis30' "
'host=***** port=***** '
"user='' password='' "
"sslmode=disable key='id' estimatedmetadata=true srid=2154 "
"type=Point checkPrimaryKeyUnicity='1' "
'table="ro"."zec_id_1_a_3499_v" (geom_2154)',
'INPUT_PROFILE': 0,
'INPUT_PROVIDER': 0,
'INPUT_START_FIELD': 'code',
'INPUT_START_LAYER': "postgres://dbname='lizmap_opensis_sis30' "
'host=***** port=***** '
"user='' password='' "
"sslmode=disable key='code' estimatedmetadata=true "
"srid=2154 type=Point checkPrimaryKeyUnicity='1' "
'table="sdis"."cis_als_v" (geom_2154)',
'OUTPUT': <QgsProcessingOutputLayerDefinition {'sink':L:/PREVISION/SIG/bd_opensis/ro/matrices/vl_h_1_cis_als_v.csv, 'createOptions': {}}>}
Traceback (most recent call last):
File "C:/Users/v-n/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\ORStools\proc\matrix_proc.py", line 189, in processAlgorithm
duration = response['durations'][s][d]
IndexError: list index out of range
L'algorithme Matrix from layers a échoué…
Execution failed after 3.84 seconds

Hello,
What kind of information do you require? I'll do my best to give them to you

Thank you very much. I've just got back to this subject. I'm finding it difficult to reproduce this behaviour systematically. The following command works but not always. Sometimes it returns an error but the output CSV is generated, correctly I think. It corresponds to what is executed in batch mode. In this mode, the execution can be correct for several entries and get stuck on a particular entry, not always the same one. By entering the rows manually or loading a JSON template, the behaviour is identical.
Thanks for your help
{"INPUT_END_FIELD":"id","INPUT_END_LAYER":"dbname='cassini' host=*************** port=**** sslmode=disable authcfg=******* key='id' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity='1' table="dni"."rop_zec_surfacique" (point_2154) sql="etat_de_l_objet" = 'En service' AND "id" < 2500","INPUT_PROFILE":0,"INPUT_PROVIDER":0,"INPUT_START_FIELD":"id","INPUT_START_LAYER":"dbname='cassini' host=*************** port=**** sslmode=disable authcfg=******* key='id' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity='1' table="dni"."org_unite_operationnelle" (geom_2154) sql="code" = 'AIM' AND "etat_de_l_objet" = 'Projeté' AND "code_du_departement" = '30'","OUTPUT":"U:/traitements/gestion_rop/matrices/aimp_to_zec2500.csv"}

Here is an extract from the JSON file, whose syntax varies slightly:
[{"PARAMETERS": {  "INPUT_PROVIDER": "0",  "INPUT_PROFILE": "0",  "INPUT_START_LAYER": "'postgres://dbname=\\'cassini\\' host=*************** port=**** sslmode=disable authcfg=******* key=\\'id\\' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity=\\'1\\' table=\"dni\".\"org_unite_operationnelle\" (geom_2154) sql=\"code\" = \\'AIM\\' AND \"etat_de_l_objet\" = \\'Projet\u00e9\\' AND \"code_du_departement\" = \\'30\\''",   "INPUT_START_FIELD": "'id'",   "INPUT_END_LAYER": "'postgres://dbname=\\'cassini\\' host=*************** port=**** sslmode=disable authcfg=******* key=\\'id\\' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity=\\'1\\' table=\"dni\".\"rop_zec_surfacique\" (point_2154) sql=\"etat_de_l_objet\" = \\'En service\\' AND \"id\" < 2500'",   "INPUT_END_FIELD": "'id'",   "INPUT_AVOID_FEATURES": "[]",   "INPUT_AVOID_BORDERS": "None",   "INPUT_AVOID_COUNTRIES": "''",   "INPUT_AVOID_POLYGONS": "None"},   "OUTPUTS": {"OUTPUT": "U:/traitements/gestion_rop/matrices/aimp_to_zec2500.csv"}},  {"PARAMETERS": {  "INPUT_PROVIDER": "0",   "INPUT_PROFILE": "0",   "INPUT_START_LAYER": "'postgres://dbname=\\'cassini\\' host=*************** port=**** sslmode=disable authcfg=******* key=\\'id\\' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity=\\'1\\' table=\"dni\".\"org_unite_operationnelle\" (geom_2154) sql=\"code\" = \\'ALS\\' AND \"etat_de_l_objet\" = \\'Actif\\' AND \"code_du_departement\" = \\'30\\''",   "INPUT_START_FIELD": "'id'",   "INPUT_END_LAYER": "'postgres://dbname=\\'cassini\\' host=*************** port=**** sslmode=disable authcfg=******* key=\\'id\\' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity=\\'1\\' table=\"dni\".\"rop_zec_surfacique\" (point_2154) sql=\"etat_de_l_objet\" = \\'En service\\' AND \"id\" < 2500'",   "INPUT_END_FIELD": "'id'",   "INPUT_AVOID_FEATURES": "[]",   "INPUT_AVOID_BORDERS": "None",   "INPUT_AVOID_COUNTRIES": "''",   "INPUT_AVOID_POLYGONS": "None"},   "OUTPUTS": {"OUTPUT": "U:/traitements/gestion_rop/matrices/als_to_zec2500.csv"}} ]

I've just run the command after loading my JSON file. Here's the error message:
Traceback (most recent call last):
File "C:\Users/********/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\ORStools\proc\matrix_proc.py", line 194, in processAlgorithm
duration = response["durations"][s][d]
IndexError: list index out of range
CSV files are generated but the last line inside is missing (normally the INPUT_END_LAYER layer should contain all id < 2500 but
id=2499 is missing). I've attached an example.
I hope I've helped.
aimp_to_zec2500.csv
als_to_zec2500.csv

One last thing. I have just run the calculation with the following parameters:
{ 'INPUT_END_FIELD' : 'id', 'INPUT_END_LAYER' : 'postgres://dbname='cassini' host=*************** port=**** sslmode=disable authcfg=******* key='id' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity='1' table="dni"."rop_zec_surfacique" (point_2154) sql="etat_de_l_objet" = 'En service' AND "id" < 2500', 'INPUT_PROFILE' : 0, 'INPUT_PROVIDER' : 0, 'INPUT_START_FIELD' : 'id', 'INPUT_START_LAYER' : 'postgres://dbname='cassini' host=*************** port=**** sslmode=disable authcfg=******* key='id' estimatedmetadata=true srid=2154 type=Point checkPrimaryKeyUnicity='1' table="dni"."org_unite_operationnelle" (geom_2154) sql="code" = 'AIM' AND "etat_de_l_objet" = 'Projeté' AND "code_du_departement" = '30'', 'OUTPUT' : 'TEMPORARY_OUTPUT' }
The result is correct, in batch mode (creating the rows by hand) or not: I have the id = 2499 in the result

Hi sigsdis30
I've had a look into this. The problem can be reproduced by popping one of the response items. That means the length of the API response is shorter that expected. Is there anything that may cause a route to not be created?
Are you still using ORS Tools v1.5.2, or the latest version? Are you using the public provider?

Hi Merydian,
Right now I'm using version 1.7.1. Previously, I had to reinstall version 1.5.2 to have no problems.
I don't see what could be causing problems. Sometimes a destination is too far from a route to be reached but, in the tests above, that's not the case. When I run outside batch mode, I get results for all cases

Hello,
The input data are two PostgreSQL tables that you cannot access. Their structure is as follows:

id serial
text code (used in INPUT_START_FIELD and INPUT_END_FIELD)
geom(Point,2154) (simple point geometry and EPSG:2154)
It's hard to get much more basic than this.

Hey,
we assume that your issue is related to the locations of the points that you're using to request your matrix with.
Thus, reproducing your issue is only possible if we have access to the geometries of the points that exhibit the problem on your side.
Could you export ids and geometry from your PostgreSQL tables and provide that?
If not, can you provide artificial data that exhibits the same problem?
Thanks for your help and patience :)

Hi sigsdis30,
may it be the case that u are using too many points at once? The restriction for the matrix API is 3500 Locations per request, see here. You can check the log after the error appears, at View>Panels>Log Messages>ORS Tools to see if this is happening.

Hello,
Thank you for taking the time to help me.
I'm attaching a geopackage containing an extract of the starting points layer and the destination points layer: https://we.tl/t-rT2tHXZTvb
There's no problem with over-quota queries, because I'm creating table extracts to make sure I don't exceed them: one view per starting point and one view per batch of 2,500 destination points. I'm running the whole thing in batch mode, which is particularly useful for managing the timeout when the number of authorised queries has been exceeded (a one-minute wait, I think, before the next burst).

Hey,
thanks for the data.
When I use this to request a matrix, I do get the error mentioned by @Merydian - the restriction is 3500 routes per request, and 2x2500 exceeds this.
I would guess that this is the underlying issue in your case as well - could you try running without views, but rather with extracted points?
The case of running with views on your psql db is quite hard to debug, since we don't have your db and views…
Please also have a look at the ORS Tools Log Messages Panel, as proposed by @Merydian , to see what issue the ors has with your query.
Best regards

You need to run separate queries for each starting point. If you don't do this, you'll exceed the quota.
Views make data management easier, but even without them, the problem occurs. I don't agree with your hypothesis

Some news?
Is it possible that the JSON syntax returned by the batch run is incorrect?
Best regards


Rename package name to work with Drupal

Addresses the package name problem described in #36.

This needs recreated against drupal8 branch.


bump dependencies

Applied mkpr on package.json
/tmp/mkpr.sh

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 91.897%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
906

Relevant Lines:
979

💛  - Coveralls


Integrate regression & forecasting models into the simulation.

Depends on
https://trello.com/c/RwZovQOp/18-dc-forecasting-notebook
https://trello.com/c/pPSMBK9R/14-dc-regression-appointments-per-person
https://trello.com/c/rVabdW4M/20-dc-load-regression-model-from-joblib-and-cache

Write into main.py

waiting on #126


Spike: Underscore behaviour in search

What are the options we have to mitigate this issue?
Issue raised with DataHub: https://datahubspace.slack.com/archives/C029A3M079U/p1714733851141439
Here's the original Slack thread in our channel which explains the issue with examples: https://mojdt.slack.com/archives/C03QZ776JVA/p1714728291635949

Raised a bug as well as poked them on Slack https://github.com/datahub-project/datahub/issues/10505

https://datahubspace.slack.com/archives/C029A3M079U/p1715761935519539

https://github.com/ministryofjustice/find-moj-data/pull/355


Learning PostgreSQL and psycopg2

In these changes, I have added PostgreSQL and psycopg2 to the Topics/Tech_Stacks.md and created a new markdown file in the path "Topics/Tech_Stacks/PostgreSQL_psycopg2.md". In this markdown file, I introduce the installation and some of the uses of PostgreSQL and psycopg2. In addition, I provided some links for further learning.

Great job! You addressed all my points in the previous review, and it is more informative and easier to follow now.

Just fixed a small formatting issue because the github pages was not correctly linking some of the table of contents. Everything is formatted correctly now.


Task: Refactor Milestone form to use the database-stored schema

Describe the task
Milestones is our most complex form.
Refactoring it will require

[x] creating a new form schema with all the fields present
[x] using oneOf to switch between milestone types
[x] writing a separate handle_commit function to translate the one form into multiple database saves

Developer info

[ ] don't forget to update the tasklist and mutation fragments to fetch the new data
[ ] existing form_change records need to be migrated with:

[ ] a migration function that goes through every existing revision, grabs the milestone reporting requirements and creates a new form_change for each of these - and delete (archive) the form_changes for reporting_requirement, payment and milestone tables.
[ ] tests for that migration function
[ ] thoughts and more testing

Additional context

Add any other context about the task here.
Or here

@pbastia, @Sepehr-Sobhani and team - congratulations! This was a huge amount of work and is quite the milestone for completion :)
Tested in dev by creating and editing projects. Observations during testing:

My mistake @pbastia and @Sepehr-Sobhani - user error. The amendments are there :)
Looks great!


Unable to use app engine Modules with NoseGAE

Hi there,
I've been trying to use more than one app engine module with NoseGAE but it doesn't seem possible.  when I run dev_appserver.py api-v1/ php-ancillary-services.yaml I am specifying firstly my project root and then my additional module, however I can't seem to find a way to do this in my functional tests?  The error I am getting is..
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/modules/modules.py", line 146, in _CheckAsyncResult
    raise mapped_error()
InvalidModuleError

I assume this is because noseGAE has no idea about my additional modules, but how would I let NoseGAE know about my additional modules?  I tried changing some of the source for NoseGAE but I can't get it working.  I don't think this'll be something I can get working myself so would appreciate any help.

I will take a look at this today as well. Can you give me the command you are using to call nosetests?

handlers section is not being used by nosegae at all, you test your code bypassing the yaml files. Only queue.yaml is properly used.

@robwithhair this is the WIP for modules usage: https://github.com/Trii/NoseGAE/pull/26
After I put together a few more tests I will make a release.

Fixed in #24


How to send folder containing files or images as artifact to a next task?

 Then Task B needs those images. Is there a way to send those images as an artifact?

Hello again @sarmientoj24! 🙂 Thanks for the question. Passing images between tasks makes dynamically provisioned volumes a good candidate, I think. I would not pass images as canonical Argo artifacts because the storage layer that stores the artifacts is quite limited. Instead, I suggest using volumes:
from hera import Workflow, WorkflowService, Task, Volume, Resources

def foo(images_path='/mnt/images'):
    import download_images

    download_images(images_path)

def bar(images_path='/mnt/images'):
    import process_images

    process_images(images_path)

ws = WorkflowService(...)
w = Workflow(..., ws)
shared_volume = Volume(mount_path='/mnt/images', size='10Gi')
foo_t = Task('foo', foo, resources=Resources(volumes=[shared_volume]))
bar_t = Task('bar', bar, resources=Resources(volumes=[shared_volume]))
foo_t >> bar_t
w.add_tasks(foo_t, bar_t)
w.create()

The above will:

create two tasks, a producer of images and a consumer of images
both mount the same dynamically provisioned volume of 10Gi
the images are stored at /mnt/images
since the volume is shared, both tasks have access to that volume via the images path
the producer can store images at that path
the consumer can consume images from that path

These are the concepts used:

https://github.com/argoproj-labs/hera-workflows/blob/main/src/hera/resources.py#L10
https://github.com/argoproj-labs/hera-workflows/blob/main/src/hera/volumes.py#L174

That is awesome @flaviuvadan
Just another question. What if I dynamically create a parallel "DAG" (say, 2). where each parallel task group should only have access to their specific image sets.
Is the following a good convention? Or will Parallel Group 2 be able to access Parallel Group 1's volume by accident?
Parallel Group 1
A_1 >> B_1 using v1 = Volume(mount_path='/mnt/images', size='10Gi')
Parallel Group 2
A_2 >> B_2 using v2 = Volume(mount_path='/mnt/images', size='10Gi')

I do not think this will happen. The volume gets a UUID assigned to it and if you share a specific volume between the tasks of a chain of tasks only those ones will get the volume attached. If you have, say, a for loop that keeps making these chains like so
w = Workflow(...)
for _ in range(5):
    r = Resources(volumes=[Volume(...)])
    a = Task(..., r=r)
    b = Task(..., r=r)
    a >> b
    w.add_tasks(a, b)

then there will be 5 chains of 2 tasks in the workflow, with each chain having access to different volumes.

Any chance you'd like to contribute some of your examples to the examples folder with some docs? 🙂 You asked about some interesting scenarios that may benefit others as well!

@flaviuvadan Awesome! I apologize for a lot of questions. Some things I can understand from reading the source code itself, some I dont really know the proper syntax. But great work.

Any chance you'd like to contribute some of your examples to the examples folder with some docs?

Sure thing. Just finishing this workflow and will contribute some things in the examples once we make it work properly.
Great work!

All questions are welcome!! 🙂 No apology needed.


Exception autowiring in MyEndpoint class

Hi! We start working with your nexrRTC framework and we have some problems with understanding.
We have service and we can't make simple autowirng it in MyEndpoint class .
Also we don't understand how to use Text event from UI.
Please connect with us as soon as possible, it's very important for us.
P.S. Please if you don't mind add me in skype - platon-kun or facebook https://www.facebook.com/
Thanks a lot!

We have service and we can't make simple autowirng it in MyEndpoint class .

What kind of service? If you're using different implementation of @ServerEndpoint, you can (instead of inheritance) just composite NextRTCEndpoint, and call appropriate method in your implementation.

Also we don't understand how to use Text event from UI.

I've comitted test which should show you how to use TEXT message.
In js code it'll looks similar to:
nextRTC.request('text', '<destination member id>', 'Hello my friend', 'custom fields (json object)');
And should be called exactly in the same way as join or create method :)

Thanks for you answer! I tried to put 'text' in request and create handler for text but I didn't catch it at all I don't know why. We have such misunderstading in logic. As we understand there are some types of event like create destroy, open and left and etc. So we need some event which you don't have at your enum, for example let it be event 'search', so we can't catch it. Maybe there is way to create own handler for custom event?
@We have service and we can't make simple autowirng it in MyEndpoint class .
My teammate ment that we have some service and would like to include in MyEndpoint class via autowiring because as I wrote above the problem with custom signal, we can get this signal only via overiding methods of NextRTCEndpoing class and get it from onMessage(Message message, Session session) now.
Thanks for answer. Again I would like to say that what you did it's great tool for a lot of business tasks!

Now your response is clear enough for me :)
I'll modify that part (I'll use normal component class instead of enum), then to add your own signal only two things will be required; create class which implements SignalHandler and register it in new class.
But up to now you can simply create class that implements SignalHandler (your signal type) and create your own SignalResolver (extends it and provide recognition for your signal for exampe
("my-signal".equals(signal) ? return new Pair("my-signal", myHandlerInstance) : super.resolve(singal)
and made your SignalResolver @Primary bean.

We decided to download your Nextrtc-signalig-server and add in Signal.class and Signals.class "search" and "searched" fields. So, from our understanding "search" is request but also we need callback so that we added "seached" field also in both classes. Also I added handler for "search" and implements SignalHandler which has method "execute" but we don't do there anything. Our search request works perfect but we can't make callback "searched" because in backend we didn't find class or interface which is responsible for calbacks. In our js we also added some things:

code below is for request to server:
NextRTC.prototype.search = function create(content, custom) {  var nextRTC = this;  nextRTC.call('search', {      content: content  });  console.log('NextRTC search method!');  nextRTC.request('search', null, content, custom); };

code below is for calback as we understand from general logic:
this.nextRTC.on('searched', content => {
 });

If we understand correct we waiting for callback from server (in our case signal name is "searched").
But we can't get it at all.
Please, if you have free time, explain us how it works at all. Because now we have stuck and we don't know what to do.
Thank you for helping before. Hope on your answer!

Hello
Please take a look into my last commit, in version 0.0.6 you'll be able to register your own handler just like that:
@Autowired
private SignalResolver resolver;

@Bean
public void addCustomHandlers(){
    resolver.addCustomHandler(Signal.fromString("search"), (msg)-> {
        InternalMessage.create()//
                .to(msg.getFrom())// here you can define where your message should be sent
                .content(id)//
                .signal("searched")//
                .build()//
                .send();
    });
}

If you want to send some message to client you can simply copy my handler

Thanks for your fix with handler. We got new strange behavior in our project. If i start sending 'create' and 'join' from the same local machine (from the same ip address) it works perfect and we see each other but when we want to connect from different machines (ip address) it starts working strange:

From one side sends signal 'create' and conversation created
Second side try to join it and it 'joined'
Windows appears but no video and no audio at first time but when do the same things again it starts working but it appears in our logic two conversations which are the same and in our logic it appears again 2 windows and in total sum of windows are 4 in ui. We can't catch this bug and dont know what to do with that.
Please if you have some opinion about this bug let me know, we would be glad to hear any versions about that.
Here some screen:

So there're two issues:

By calling create without conversation name your're creating two
conversations (that'll be fixed on the server side -> one peer will be able
to create or join to only one conversation)

In broadcast conversation (in the past it was working fine but this
behavior wasn't described in WebRTC documentation draft), I was filtering
candidates coming from audiences. That allows me to make only one direction
streaming in global network (broadcast was designed to have 1 broadcaster
and N listeners). Now It seems that when candidates from audience peer will
not be delivered to broadcaster streaming won't be started at all.

To omit this problems (up to fix ;) ), you can simply use conversation type
MESH, and provide conversation name for it. MESH conversation was designed
to make a call between group of people and each will have a direct
streaming to the rest.
If that workaround is not enough for you, you have to wait to the Monday
for fix.
2017-02-17 16:22 GMT+01:00 David<EMAIL_ADDRESS>
Thanks for your fix with handler. We got new strange behavior in our
project. If i start sending 'create' and 'join' from the same local machine
(from the same ip address) it works perfect and we see each other but when
we want to connect from different machines (ip address) it starts working
strange:

From one side sends signal 'create' and conversation created
Second side try to join it and it 'joined'
Windows appears but no video and no audio at first time but when do
the same things again it starts working but it appears in our logic two
conversations which are the same and in our logic it appears again 2
windows and in total sum of windows are 4 in ui. We can't catch this bug
and dont know what to do with that.
Please if you have some opinion about this bug let me know, we would
be glad to hear any versions about that.
Here some screen:
[image: i23 cimgpsh_orig_ink_li]
https://cloud.githubusercontent.com/assets/13423531/23071014/67ecabcc-f546-11e6-96df-9d65b8a8b4a8.jpg
[image: i24 cimgpsh_orig_ink_li]
https://cloud.githubusercontent.com/assets/13423531/23071015/67edb210-f546-11e6-97c9-38fd9588a67a.jpg

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mslosarz/nextrtc-videochat-with-rest/issues/7#issuecomment-280678512,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADysRfJRIqAQcb9rJHl2064kbN9qk9Joks5rdbs2gaJpZM4MCMbL
.

--
Marcin Ślósarz

Thanks you for quickly response. You really helped us with understanding. Just right now we changed broadcast to mesh and it started working perfect! Thanks for your replies at all. We noticed that you're very talented guy. I think everything what is rest we can do it by ourselves becasue networking with sockets are end. :)

Broadcast is fixed you can find it in 0.0.6-SNAPSHOT. I'll release this version soon

Are you still developing and maintaining this project?


Environmental Controller with peace module doesn't stop bosses from spawning

Not sure how they prioritize their event hook but hopefully this can somehow be fixed. Maybe I need to file the bug with them instead? Let me know.
The code in rftools that should handle this seem to be here. It's definitely working on normal world spawns, and as long as you're creating that boss arrival event using w/e would trigger this hook it SHOULD work i imagine :(.
https://github.com/McJtyMods/RFTools/blob/030a495c8530183bdfe47475ed41d45386e56f6c/src/main/java/mcjty/rftools/ForgeEventHandlers.java#L228

The linked code looks like its for a very very old version of MC (~1.12).  I'm not sure where the equivalent code for modern versions is.  It would appear to be https://github.com/McJtyMods/RFToolsUtility/blob/1.20/src/main/java/mcjty/rftoolsutility/setup/ForgeEventHandlers.java#L245 but this doesn't look like it gets called anywhere.

I don't believe there are any changes I can make to assist in this process.  Apoth listens to the FinalizeSpawn event for triggering these spawns, which should be cancelled by the controlled.


generate kafka.bnf from kafka client code

@zmstone let me know what do you think about this

brilliant! PR the new class to kafka?
They have a toHtml, why not a toText to make everyone else's life easier.

Good idea, will do later today.

So, I've been trying to finish that PR, with a goal to support kafka 0.10.0+ versioned API in brod and streamline updating kafka_protocol in future. Some observations so far:
There's unwielding amount of records with the same name but different structure in generated code. Previously clashes were resolved by hand, with records named like #kpro_OCReqV2Topic and keys like oCReqV2Partition_L, but this won't work for automated export of 'BNF' from  apache/kafka. Naive shortening would fail on LeaveGroupRequest and ListGroupsRequest, for example. With almost identical subrecords in every version of each request, we probably need to prefix every subrecord name. Like this:
-record( kpro_MetadataResponseV0, {brokers, topicMetadata}).
-record( kpro_MetadataResponseV0_Broker, {nodeId, host, port}).
-record( kpro_MetadataResponseV0_TopicMetadata, {topicErrorCode, topic, partitionMetadata}).
-record( kpro_MetadataResponseV0_PartitionMetadata, {partitionErrorCode, partitionId, leader, replicas, isr}).

-record( kpro_MetadataResponseV2, {brokers, clusterId, controllerId, topicMetadata}).
-record( kpro_MetadataResponseV2_Broker, {nodeId, host, port, rack}).
-record( kpro_MetadataResponseV2_TopicMetadata, {topicErrorCode, topic, isInternal, partitionMetadata}).
-record( kpro_MetadataResponseV2_PartitionMetadata, {partitionErrorCode, partitionId, leader, replicas, isr}).

Btw, while we're at this and will have to rename things in brod anyway, and as original BNF is using under_score instead of camelCase, I'd like to rename everything in idiomatic erlang style. What do you think? Something like this:
-record( kpro_metadata_response_v2, {brokers, cluster_id, controller_id, topic_metadata}).
-record( kpro_metadata_response_v2_broker, {node_id, host, port, rack}).
-record( kpro_metadata_response_v2_topic_metadata, {topic_error_code, topic, is_internal, partition_metadata}).
-record( kpro_metadata_response_v2_partition_metadata, {partition_error_code, partition_id, leader, replicas, isr}).

anyway, #kpro_MetadataResponseV0_PartitionMetadata and #kpro_MetadataResponseV2_PartitionMetadata have the same structure between versions, but ..TopicMetadata.. is changing between v1 and v2. Having different versioning for subrecords is not an option, I think, but using the same versioning as the parent record would be quite painful/verbose to construct and match in brod.
But structs "evolve" quite slow and conservatively. I made a small repository https://github.com/codesnik/kafka_structs_versions , git log -p --color-words on it is quite illustrative.
What if I autogenerate "union" records for every possible request and response in a particular API endpoint, with undefined for deprecated or unused fields? A bit too explicit pseudocode:
-record( kpro_metadata_response, {brokers, cluster_id=undefined, controller_id=undefined, topic_metadata}).
-record( kpro_metadata_response_broker, {node_id, host, port, rack=undefined}).
-record( kpro_metadata_response_topic_metadata, {topic_error_code, topic, is_internal=undefined, partition_metadata}).
-record( kpro_metadata_response_partition_metadata, {partition_error_code, partition_id, leader, replicas, isr}).

then, encode function would also get Version argument to delegate encoding of the record to a proper encoder, and will also pass that version to subrecord encoders. Actual Version range could be  restricted in brod by ApiVersions handshake response and further guessed by matching on defined parameter list. This probably would hurt static analysis, though I don't know if and how you use it in brod. I don't know if I'm being clear enough, but any comments for that?
miscelaneous:
We can probably use 2 or 3 bits more from correlation_id for response version in addition to 5 bits for API number. Different brokers could have different kafka versions on them even in one cluster, so it's probably the easiest solution.
There's nullable_string type now in BNF. Current kpro:str() mapped to string BNF type accepts undefined and dumps -1 as the size, but only nullable_string should be able to accept undefined, according to kafka sources. Should I just alias nullable_string to kpro:str(), or should I define a new type? Restricting string could cause sudden runtime errors on message serialization. Or not.

Thanks for the hard work!
Replying the easy ones first:

camelCase was derived from 0.8/0.9 specs, I'm totally for snake_case in new version
I don't remember if there is a case where we MUST use a null, if not, we can produce ONLY empty strings when it's 'undefined' for example.
I once ran dialyzer check on brod, but never succeeded making it 100% happy. But if there is a much nicer solution will less nicer static type info, don't get your hands tied.
Yes, let's use 8 bits for API-KEY + version to keep it simple.

With regards to versioning, as long as there is no exponential combination of name + version + sub-name + sub-version, I'm for enumerating them.
We can create APIs on top to make record construction easier.
And the decode result can be translated to nicer representation after decoding.
e.g. convert record to a map()
Making it easier at low level (e.g. merge fields to make a union record, or use map())
will make it harder at higher level when constructing
the requests. e.g. one will have to remember which field should be set for which version
if we have clean records, compiler should check it for us.

Awesome stuff. Thanks for finishing it.
Can't approve since I'm the original author, so just go ahead and merge it.

Approved.

will merge when brod 3.0 development is ready to release.


Only use output struct values when actually generated

This PR is an approach to improving data quality for data generated by preprocess
adapters. The PR updates the generated code for Output structs for preprocess adapters
by adding SetXXXX() methods to the Output type that keep track of attributes that
were set. The generated code that checks for attribute values then also checks to see
if an attribute value was set.
In support of those changes, this PR adds a new expr Fn to Mixer: emptyStringMap(). This
new expr allows creation of default values for string_map expressions.
Finally, the PR updates the various config components for kubernetesenv adapters throughout
the repo.
This should eliminate the source.service == "" data issues that appeared when the new preprocessor
adapter mechanisms were put in place (as well as other zero-value related issues).

/lgtm

@ozevren this PR merged at light-speed. I was halfway through implementing the requested tests and BOOM! it was merged. Amazing. I'll send a follow-up PR.
I did notice something in testing however. It appears that externs must always appear in the right-most position of an expression to work as expected. To clarify, for the expression:
emptyStringMap() | source.labels
I would expect that the result would be map[string]string{} with no referenced attributes.
What I observed instead, however, was the result was whatever was specified in source.labels and source.labels was a referenced attribute.
Checking the generated IL, I saw something like:
func ... {
   call emptyStringMap
   resolve_f "source.labels"
   ret
}

Is that WAI?  It seems like a potential bug.

this PR merged at light-speed.
it's the new vendor submodule :-)


chore: rename gravity module

Related Github tickets

https://github.com/VolumeFi/paloma/issues/1801

Background
As a Paloma user, I want the "gravity2" module to have a name that better represents it's functionality (it was heavily modified from the original gravity source) and I want the (cli) documentation to make clear that it is multichain instead of just for ETH.

[!CAUTION]
This change is breaking and will remove any data stored within the Gravity store.

Testing completed

[x] test coverage exists or has been added/updated
[x] tested in a private testnet

Breaking changes

[x] I have checked my code for breaking changes
[x] If there are breaking changes, there is a supporting migration.

Wow. That was big and fast! Congrats @byte-bandit


spark_connect failed on OS X:Failed to launch Spark shell. Ports file does not exist.

> library(sparklyr)
> library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

> sc <- spark_connect(master = "local")
Error in sparkapi::start_shell(master = master, spark_home = spark_home,  :
  Failed to launch Spark shell. Ports file does not exist.
    Path: /Users/13k/Library/Caches/spark/spark-2.0.0-preview-bin-hadoop2.6/bin/spark-submit
    Parameters: --jars, '/Users/13k/R/sparklyr/java/rspark_utils.jar', --packages, 'com.databricks:spark-csv_2.11:1.3.0','com.amazonaws:aws-java-sdk-pom:1.10.34', sparkr-shell, /var/folders/sd/2_0mf8p100v7dl3p_2cs5m_80000gn/T//RtmpXUplRl/file1323016779f8e.out

Ivy Default Cache set to: /Users/13k/.ivy2/cache
The jars for the packages stored in: /Users/13k/.ivy2/jars
:: loading settings :: url = jar:file:/Users/13k/Library/Caches/spark/spark-2.0.0-preview-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
com.databricks#spark-csv_2.11 added as a dependency
com.amazonaws#aws-java-sdk-pom added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
	confs: [default]
Exception in thread "main" java.io.FileNotFoundException: /Users/13k/.ivy2/cache/resolved-org.apache.spar

sessioninfo:
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.4 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] dplyr_0.5.0     sparklyr_0.2.24

loaded via a namespace (and not attached):
 [1] magrittr_1.5    R6_2.1.2        assertthat_0.1  rsconnect_0.4.3
 [5] parallel_3.3.1  DBI_0.4-2       tools_3.3.1     withr_1.0.2
 [9] yaml_2.1.13     rappdirs_0.3.1  tibble_1.1      Rcpp_0.12.5
[13] config_0.1.0    sparkapi_0.3.12

and i tested sparklyr in terminal with sudo R, and it works well.
it's a permission issue about RStudio or something else?

The error makes me think that this could be related with loading spark packages; therefore, I would try connecting without any additional packages as follows:
sc <- spark_connect(master = "local", config = list())

Would this work for you?

@javierluraschi yea it can connect by using your code but when i try copy_to it said Error: could not find function "copy_to" and load dplyr package then copy_to, pop another error Error: java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.csv

@funnng this scopes the problem to loading spark packages. The most common cause would be that something is blocking downloads for that user.
That said, a fix included in 0.2.28 that will allow you to perform copy_to even when packages are not loaded. However, not loading packages will decrease copy_to performance significantly, but we are also working on improving this in the near future to avoid requiring additional packages.

I had a similar issue while using the CDH-provided Spark, resolved when I switched to a Databricks prebuilt distro. I got some interesting error messages when trying the arfument config==list(), as suggested above. You may want to have a look at the full details in this SO thread (used version 0.2.28).

@desertnaut @funnng sparklyr version 0.2.29 adds support for CDH. CDH is currently under testing on our end, but if you are interested, please feel free to go ahead and give this a try.

Thanx - did 0.2.29 just come out? I installed it yesterday, and I got 0.2.28.

@desertnaut Yes, just bumped the version with this commit https://github.com/rstudio/sparklyr/commit/1726b24466ce1cc9e296bebb80d2b7de12f1f966. Although the relevant commit is this one https://github.com/rstudio/sparklyr/commit/036101f35d1757bd53f4f241e12973fd53b28de0 but it's hard to tell if you have the changes without a version bump.

OK, I'll take the new version for a test drive with CDH asap. BTW, thanks for this great work...!

@desertnaut there were a couple additional issues that affected the 2.0.0-preview version. If you were using this version, then you would want to upgrade sparklyr; otherwise, you can ignore this comment.

I'm also running into this issue

devtools::install_github("rstudio/sparklyr",dependencies =TRUE)

library(sparklyr)
spark_install(version = "1.6.2")

# Installation complete.
# Warning message:
#   In value[[3L]](cond) : Failed to set logging settings

library(sparklyr)
sc <- spark_connect(master = "local",config = list())
# Error in sparkapi::start_shell(master = master, spark_home = spark_home,  :
#                                  Failed to launch Spark shell. Ports file does not exist.

# iris_tbl <- copy_to(sc, iris)

R version 3.2.3 (2015-12-10)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS release 6.7 (Final)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] sparklyr_0.2.31

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.6     sparkapi_0.3.22 digest_0.6.9    dplyr_0.5.0     withr_1.0.2
 [6] rprojroot_1.0-2 assertthat_0.1  rappdirs_0.3.1  R6_2.1.2        DBI_0.4-1
[11] git2r_0.14.0    magrittr_1.5    httr_1.2.1      curl_1.1        config_0.1.0
[16] devtools_1.10.0 tools_3.2.3     readr_0.2.2     parallel_3.2.3  yaml_2.1.13
[21] memoise_1.0.0   tibble_1.1

On mac I am experiencing the same issue but only when I edit the config.yml file. In particular, it is the final line here that is throwing the error:
default:
  sparklyr.cores.local: 8
  sparklyr.shell.driver-memory: 4GB

I am getting this error on Mac.. What seems to be the problem?
library(sparklyr)
library(dplyr)
spark_install_dir()
[1] "/Users/selvaprabhakaran/Library/Caches/spark"
Sys.setenv(SPARK_HOME="/Users/selvaprabhakaran/Library/Caches/spark")

sc <- spark_connect(master = "local", config = list(), version = "1.6.1")
Error in start_shell(master = master, spark_home = spark_home, spark_version = version,  :
  Failed to launch Spark shell. Ports file does not exist.
    Path: /Users/selvaprabhakaran/Library/Caches/spark/bin/spark-submit
    Parameters: --class, sparklyr.Backend, '/Library/Frameworks/R.framework/Versions/3.3/Resources/library/sparklyr/java/sparklyr-1.6-2.10.jar', /var/folders/q5/gkgbtsnx0mjg2rq_0bdwct4c0000gp/T//RtmpgUvlqC/file127764421f8b1.out

sh: /Users/selvaprabhakaran/Library/Caches/spark/bin/spark-submit: No such file or directory
In addition: Warning message:
In normalizePath(file.path(spark_home, "bin", spark_submit)) :
  path[1]="/Users/selvaprabhakaran/Library/Caches/spark/bin/spark-submit": No such file or directory

sessioninfo:
sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.10.5 (Yosemite)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] sparklyr_0.3.3

loaded via a namespace (and not attached):
 [1] readr_1.0.0     magrittr_1.5    R6_2.1.2        assertthat_0.1
 [5] rprojroot_1.0-2 parallel_3.3.1  DBI_0.4-1       tools_3.3.1
 [9] withr_1.0.2     dplyr_0.5.0     rappdirs_0.3.1  yaml_2.1.13
[13] tibble_1.1      Rcpp_0.12.6     config_0.2      digest_0.6.10

Turns out there was 2 versions of spark in my SPARK_HOME="/Users/selvaprabhakaran/Library/Caches/spark"
So, I set it to:
Sys.setenv(SPARK_HOME="/Users/selvaprabhakaran/Library/Caches/spark/spark-1.6.1-bin-hadoop2.6")

But, now I get different exceptions..
> sc <- spark_connect(master = "local", config = list(), version = "1.6.1")
Error in start_shell(master = master, spark_home = spark_home, spark_version = version,  :
  Failed to launch Spark shell. Ports file does not exist.
    Path: /Users/selvaprabhakaran/Library/Caches/spark/spark-1.6.1-bin-hadoop2.6/bin/spark-submit
    Parameters: --class, sparklyr.Backend, '/Library/Frameworks/R.framework/Versions/3.3/Resources/library/sparklyr/java/sparklyr-1.6-2.10.jar', /var/folders/q5/gkgbtsnx0mjg2rq_0bdwct4c0000gp/T//RtmpgUvlqC/file127767a2bc95b.out

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/launcher/Main
Caused by: java.lang.ClassNotFoundException: org.apache.spark.launcher.Main
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

Resolved it by re-installing JDK 8 from this link looks like there was a conflict with the Java version used by terminal and by setting Sys.setenv(SPARK_LOCAL_IP="<IP_ADDRESS>")

hi,
i am trying to connect to emr master server running yarn and getting this error -- any help?

sc <- spark_connect(master = "yarn-client")
Error: Failed during initialize_connection() java.net.ConnectException: Call From 98bda6519109/<IP_ADDRESS> to <IP_ADDRESS>:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy10.getClusterMetrics(Unknown Source)
at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImp

@mkaja looks like there is no access from the machine where the r-session is running to the emr driver node. If you are running in emr, I would recommend you install rstudio in emr. For instance, see: https://blogs.aws.amazon.com/bigdata/post/Tx37RSKRFDQNTSL/Statistical-Analysis-with-Open-Source-R-and-RStudio-on-Amazon-EMR

Hi @selva86, could you tell me which version of JAVA did you re-installed, because I also have the same problem that you met before.
I am on windows 10
my java  version is

java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

@JackHo327 I installed 'jdk-8u101-macosx-x64.dmg' from link. You might've to install 'jdk-8u101-windows-x64.exe' if your PC is 64bit.

Thank you @selva86 .
but unfortunately, still got that problem.
so strange because I can run this at spark 1.6.x

@JackHo327 how are you setting the SPARK_HOME? sparklyr looks at the system environment variable SPARK_HOME so if this happens to point to the 1.6.* version, it would also use the wrong jars while deploying to Spark.

Sys.setenv(HADOOP_CONF_DIR= "/etc/hadoop/conf")
Sys.setenv(SPARK_HOME=
"/home/user-id/.cache/spark/spark-1.6.0-bin-hadoop2.6")
On Mon, Sep 26, 2016 at 2:42 PM, Javier Luraschi<EMAIL_ADDRESS>wrote:

@JackHo327 https://github.com/JackHo327 how are you setting the
SPARK_HOME? sparklyr looks at the system environment variable SPARK_HOME
so if this happens to point to the 1.6.* version, it would also use the
wrong jars while deploying to Spark.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rstudio/sparklyr/issues/114#issuecomment-249675604,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL8FApPoNNznXswZNPyaxxwMG8Bf5PqGks5quCAvgaJpZM4JOMSH
.

@selva86 I have two installed on Mac as well. Solved by deleting the brew installed one and restart the Mac.

The connection code has been significantly simplified starting with sparklyr version 0.4.9.
Please upgrade sparklyr using devtools::install_github("rstudio/sparklyr") and report back if any connection issues arise.

@javierluraschi  my problem solved wi 0.4.9 version.
ps. need to point out the spark.sql.warehouse.dir

Just a note on this from me. Using the CRAN version (0.4 at time of writing) and specifying the config = list() option as advised above was not loading the necessary csv spark packages and hence it was not possible for me to read a CSV from hdfs into a spark dataframe with spark_read_csv.
Installing the dev. version from Github solved this for me. I'm on Spark 1.6.0.


docs: format rsbuild zh chapter

Description
format rsbuild zh chapter.

Related Issue

Types of changes

[x] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

Checklist

[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] I have updated the documentation.

Thank you!


Add video from Kubecon to readme

Issue #, if available:
Description of changes: Adding video from Kubecon
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #79 (226296c) into main (1a831be) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #79   +/-   ##
=======================================
  Coverage   61.44%   61.44%
=======================================
  Files          13       13
  Lines        1149     1149
=======================================
  Hits          706      706
  Misses        403      403
  Partials       40       40

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a831be...226296c. Read the comment docs.


.form-inline used as mixin in navbar component

I tried to recompile a custom distribution without the forms components (commented out @import "forms.less"; and @import "input-groups.less"; from the main bootstrap less file) and a fatal error occurred due to .form-inlineclass  being used in the navbar.less component.
Bug:
Class .form-inline defined in less/forms.less is used in less/navbar.less as a mixin. Trying to compile the bootstrap.less without the forms components (forms.less and input-groups.less) triggers a fatal error.
Steps to reproduce:

Download and npm install the twitter bootstrap master branch.
Comment out forms.less and input-groups.less from the main bootstrap.less file.
grunt less should fail with the following error:

>> NameError: .form-inline is undefined in less/navbar.less on line 296, column 3:
>> 295   // Mixin behavior for optimum display
>> 296   .form-inline();

Resolution proposal:
Consider moving the code from the .form-inline class declaration into a mixin in the appropriate file: less/mixing/forms.less.
This should break this existing dependency and allow to compile a custom distribution without the forms components (mentioned above).

Closing per #19432.


Cant open it

It says this plz help:
C:\Users\Diego\Desktop\shopify-monitor-master>npm start

<EMAIL_ADDRESS>start C:\Users\Diego\Desktop\shopify-monitor-master
node server.js

internal/modules/cjs/loader.js:1170
throw err;
^
SyntaxError: C:\Users\Diego\Desktop\shopify-monitor-master\config.json: Unexpected token . in JSON at position 961
at parse ()
at Object.Module._extensions..json (internal/modules/cjs/loader.js:1167:22)
at Module.load (internal/modules/cjs/loader.js:977:32)
at Function.Module._load (internal/modules/cjs/loader.js:877:14)
at Module.require (internal/modules/cjs/loader.js:1019:19)
at require (internal/modules/cjs/helpers.js:77:18)
at Object. (C:\Users\Diego\Desktop\shopify-monitor-master\server.js:31:17)
at Module._compile (internal/modules/cjs/loader.js:1133:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
at Module.load (internal/modules/cjs/loader.js:977:32)
at Function.Module._load (internal/modules/cjs/loader.js:877:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
at internal/main/run_main_module.js:18:47
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: node server.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      C:\Users\Diego\AppData\Roaming\npm-cache_logs\2020-05-13T03_09_04_480Z-debug.log

Maybe you wrote wrong your json file, maybe you leave a comma or bracket that you need to delete

Maybe you wrote wrong your json file, maybe you leave a comma or bracket that you need to delete

Hey, finally manage to turn on, it gets stucked on info conection done and then the site is that normal? also is there a way to only use localhost as proxy i only want to monitor 3 sites :) Thx

yes, the monitor notify you online error or worning, so you see only the info connection when it start
if you receive the connection info from the cmd and the notify in discord, means that the monitor is working
if you want to use only local ip, don't insert any proxy ip, and the monitor will use automatically your local ip


how to I self host viz.js?

I have installed viz.js using npm install on my rpi 2 box.
But how do I host it to access it in browser.

is this project still maintained ?

Try with this: https://github.com/matkokvesic/express-graphviz

Thanks @matkokvesic.
I am getting "Error: syntax error in line 1 near '򀀀" error while running it.
Any idea how to fix it ?

Thanks @matkokvesic :+1:


Add  AlibabaCloud LogService  to vendor list

Changes
Please provide a brief description of the changes here.
Merge Requirements
For new features contributions please make sure you have completed the following
essential items:

[ ] CHANGELOG.md updated to document new feature additions
[ ] Appropriate documentation updates in the docs folder

Maintainers will not merge until the above have been completed. If you're unsure
which docs need to be changed ping the
@open-telemetry/demo-approvers.

The list is ordered alphabetically, could you update it to add AlibabaCloud LogService as 1st entry?

Done

Thanks for your contribution!


Fix the build for #15013.: Lookup jitter upstream build fix

Fix the build for https://github.com/apache/druid/pull/15013.
This PR has:

[ ] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

Fixing  the build for https://github.com/apache/druid/pull/15013 which was approved already.


fix: duplicate name/label values

This addesses #73
Signed-off-by: Kevin Belisle<EMAIL_ADDRESS>

Would need to update the example metrics in the readme as well.

An alternative would be to add the statistic's namespace as a label.
Namespace being the term used on the wiki, unclear if this is Mojang's official term for it.

closed in favour of #77


Update README

Since 5d19e845eb8b5c7e9df1cae9d6180f9bf7840613 we are now using C++14

Lint fail is related to issue #225, not related to these patch


Fix ng2-dnd issues on Android and IOS devices

Seems, there issues with TouchEvent in Angular2.

The problem is, the TouchEvents are fired well as expected but there is no VISUAL drag and drop. I think we have to launch manually the dragstart event or something like that. Do you know, is there are possibility to start dragging manually? If you must manage the dragging with a canvas and check if the canvas is before after or over a other drag enbaled object it would heavier to resolve this problem? What do you think?

Possible solution - use  the following shim in your project:
https://github.com/timruffles/ios-html5-drag-drop-shim
Can you check does it work for you?

Hey Sergey,
should it not possible to dispatch a DragEvent doing something like that?
var dragEvent:DragEvent = document.createEvent('DragEvent');
dragEvent.initDragEvent("dragstart",
touchEvent.bubbles,
touchEvent.cancelable,
touchEvent.view,
touchEvent.detail,
this.laDraggableService.touchStartX,
this.laDraggableService.touchStartY,
this.laDraggableService.touchStartX, //
this.laDraggableService.touchStartY, //
touchEvent.ctrlKey,
touchEvent.altKey,
touchEvent.shiftKey,
touchEvent.metaKey,
0,
touchEvent.target,
new DataTransfer());
this.elemRef.nativeElement.dispatchEvent(dragEvent:DragEvent);
like this?
//var textEvent = document.createEvent('TextEvent');
//textEvent.initTextEvent('textInput', true, true,
null, '\0', 9, "en-US");
//this.elemRef.nativeElement.dispatchEvent(textEvent);
For now this is not working for me ... but I'm working on it, what do you
think?
2016-04-18 7:42 GMT+02:00 Sergey Akopkokhyants<EMAIL_ADDRESS>
Possible solution - use the following shim in your project:
https://github.com/timruffles/ios-html5-drag-drop-shim
Can you check does it work for you?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-211213166

Seems you try to initialise DragEvent with TouchEvent properties. I don't think it is possible.
The question to you - did you check this shim. Does it work for you? If it does - I can incorporate it inside ng2-dnd with improvements.

No I didn't check it, I'am not so happy to use an other external LIB in JS
to resolve it. I don't know if it there is a possibility to get it work,
but in my opinion it should work natively in angular2. If I have to inlcude
an external JS I can also include JQUERY UI and anything works great there.
But I will test no the shim.
p.s. I never dispatch an Event on javascript or angular2 so I'm not sure
what there is to do.
I controlled the DragEvent on Desktop and the TouchEvent on mobile and
there are al properties I need exptect then TARGET and DataTransfer.
I get this error
EXCEPTION: TypeError: dragEvent.initDragEvent is not a function
2016-04-18 11:54 GMT+02:00 Sergey Akopkokhyants<EMAIL_ADDRESS>
Seems you try to initialise DragEvent with TouchEvent properties. I don't
think it is possible.
The question to you - did you check this shim. Does it work for you? If it
does - I can incorporate it inside ng2-dnd with improvements.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-211304735

I think those resources could help you to realise you ideas:

https://mobiforge.com/design-development/touch-friendly-drag-and-drop
http://mobiforge.com/design-development/html5-mobile-web-touch-events

I spent the rest of Sunday to made it works but I didn't finish incorporation yet. Will continue on this week.

To make it works with the shime code or nativley?
2016-04-18 12:16 GMT+02:00 Sergey Akopkokhyants<EMAIL_ADDRESS>
I think those resources could help you to realise you ideas:

https://mobiforge.com/design-development/touch-friendly-drag-and-drop
http://mobiforge.com/design-development/html5-mobile-web-touch-events

I spent the rest of Sunday to made it works but I didn't finish
incorporation yet. Will continue on this week.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-211313149

ios-drag-drop.js works but in a very bad way.
The two links you send me I found weeks ago and I tried to implement the
normal JS solution natively in  angular2 but I stopped on the first
problem. I was not able to copy my DIV element and attache it somewhere in
the DOM. We are new on angular so there are sometimes howto problems for
us. :|
2016-04-18 12:19 GMT+02:00 Michael Burger<EMAIL_ADDRESS>
To make it works with the shime code or nativley?
2016-04-18 12:16 GMT+02:00 Sergey Akopkokhyants<EMAIL_ADDRESS>:

I think those resources could help you to realise you ideas:

https://mobiforge.com/design-development/touch-friendly-drag-and-drop
http://mobiforge.com/design-development/html5-mobile-web-touch-events
I spent the rest of Sunday to made it works but I didn't finish
incorporation yet. Will continue on this week.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-211313149

I was able to create a new Click Event with DOM Adapter and dispatch the
event for my element.
But I found no way to initDragEvent ....
2016-04-18 12:46 GMT+02:00 Michael Burger<EMAIL_ADDRESS>
ios-drag-drop.js works but in a very bad way.
The two links you send me I found weeks ago and I tried to implement the
normal JS solution natively in  angular2 but I stopped on the first
problem. I was not able to copy my DIV element and attache it somewhere in
the DOM. We are new on angular so there are sometimes howto problems for
us. :|
2016-04-18 12:19 GMT+02:00 Michael Burger<EMAIL_ADDRESS>
To make it works with the shime code or nativley?
2016-04-18 12:16 GMT+02:00 Sergey Akopkokhyants<EMAIL_ADDRESS>
:

I think those resources could help you to realise you ideas:

https://mobiforge.com/design-development/touch-friendly-drag-and-drop
http://mobiforge.com/design-development/html5-mobile-web-touch-events
I spent the rest of Sunday to made it works but I didn't finish
incorporation yet. Will continue on this week.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-211313149

I am not able to make any drag-n-drop features to work on any browsers for mobile devices. Is there any plan fixing that issue or the lib is created for desktop purposes only ?

It is not for desktop purposes only, and I will accommodate it for mobiles as well, but a bit later, when to fix all issues.

@akserg When can you fix this issue. I think this is a long due bug since April.

@akserg Tried https://github.com/timruffles/ios-html5-drag-drop-shim, neither ios nor android works.

Hi, @akserg did you evere implement drag&drop for mobile touch events?

Hi @akserg, do you plan to implement dnd for mobile?

Hi guys,
As a workaround, i am using https://github.com/timruffles/ios-html5-drag-drop-shim
npm install --save drag-drop-webkit-mobile
Then, in .angular-cli.json
"scripts": [         "../node_modules/drag-drop-webkit-mobile/ios-drag-drop.js"       ],
I can't figure out how to configure this library inside my SharedModule, so in index.html
  <script>     var iosDragDropShim = { requireExplicitDraggable: true }   </script>
And bingo! Drag and Drop is now working. Some bugs are popping up for me, but it's not breaking the app.

For Android devices you can use this workaround
.yourDivName{ touch-action: none; }
[https://github.com/akserg/ng2-dnd/issues/113#issuecomment-295360235](credits @PierreDugue)

I made the touch behaviour work with mobile-drag-drop (which is a rewrite of ios-html5-drag-drop-shim mentioned above). (Is working in emulator. Gonna test on an android device in the morning).
Angular project created with @angular/cli. Webpack.
Setup:
npm install --save 'mobile-drag-drop'

in your ./src/styles.(s)css add
@import "~mobile-drag-drop/default.css";

in your ./src/app/[drag-test].component/[drag-test].component.ts add
// imports
import { polyfill } from 'mobile-drag-drop';
...
@Component(...)
export class DragTestComponent implements OnInit {
  ...
  // constructor
  constructor() {
    polyfill({});
  }
  ...
  // prevent default on some drag events on dnd-droppable (important!)
  public preventDefault(event) {
    event.mouseEvent.preventDefault();
    return false;
  }
}

In path/to/[drag-test].component.html execute the above preventDefault($event) on elements with dnd-droppable directive:
<div dnd-droppable
  (onDragEnter)="preventDefault($event)"
  (onDragOver)="preventDefault($event)"
  (onDragLeave)="preventDefault($event)">draggables can be dropped here</div>

Does it make sense?

@akserg did you ever go ahead on this problem?

Hi there,
I will have a look.
On 8 February 2018 at 17:31, mburger81<EMAIL_ADDRESS>wrote:

@akserg https://github.com/akserg did you ever go ahead on this problem?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/akserg/ng2-dnd/issues/9#issuecomment-364147972, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABVeWcWQP4h1KND5nvxVkdESjGE0sNhmks5tSxM9gaJpZM4H-zug
.

@akserg looking to your AbstractComponent it seems you are using native drag&drop right?
Most drag&drop libraries append a cloned node to body and don't use browsers drag and drop, in this way you can use also touch events and do the same as you do on browser. But this could bring several problems, like style or performance problems!
Using hammerjs for touch events is nothing special, we use it already in our ionic projects.
So do you think you can listen for touch events and simulate native browser dragging? We tried do implement this in several ways but we wasn't able to do this, and we think this is impossible.

Any news about mobile support? I didn't manage to get the workarounds to work for me.

@akserg perhapse we can help you in some way?

hello everyone, i was struggle trying to get this plugin work for half a day as well, and eventually i could find another library that does the hard work. Check out https://github.com/SortableJS/angular-sortablejs
To get it working on Angular 2 app, please install angular-sortablejs 2.1.0 + sortablejs 1.6.0 (I spent another hour finding this answer....)
For Angular 4 app, just follow documentation

@mrkvon made perfect sense and it worked perfectly!
just to add a little detail, don't forget to add 'touch-action: none' on the div you are using to prevent the scroll from working while dragging!


.include line missing from a unit file after submitting it to fleet

I wanted to create a nice common unit file that I will include with all docker units, so I don't have to repeat myself when writing units for fleet. However, the .include line seems to be missing from the resulting unit file.  When communicating over the API unit files are deserialized (via go-systemd) into a sequence of unit options, which means we lose any comments and other things like includes. (I also seem to recall that .include directives are strongly discouraged by upstream systemd, but can't find a reference for that now.)
Not sure how to go about approaching this one.


New compliance test to ensure types can be retrieved from server.

Part of https://jira.odpi.org/browse/EGERIA-11
Signed-off-by: Mandy Chessell<EMAIL_ADDRESS>

Builds clean. Can connect to OMAG Application, and test tool launches ok. Looks like a step forward so ok to commit to master


wording

When liana download a new bitcoin core version it should be nice to notice user what we are downloading:

Should we replace 'downloading' by 'downloading bitcoin core vx.x' ?

fix by #912


Dropdown: Add backdrop for mobile dropdown popover

Description
The mobile dropdown is currently missing the backdrop behind the popover:

Who will benefit from this feature?
No response
Anything else?
No response

Completed in #1016


f1 key on type does not query reference documentation.

Assuming the following code:
let combine strs = System.String.Concat(Array.ofList(strs) : string[])

Hit the f1 key on Concat.
Expected: Browser tab (en-us) open with this link: https://msdn.microsoft.com/en-us/library/0wkb0y3w(v=vs.110).aspx
Actual: This link is opened instead: https://docs.microsoft.com/en-us/visualstudio/ide/creating-solutions-and-projects

This is the interface which should be implemented: https://github.com/dotnet/roslyn/blob/master/src/VisualStudio/Core/Def/Implementation/F1Help/IHelpContextService.cs
Spoke with @CyrusNajmabadi and he said that string FormatSymbol(ISymbol symbol); can just return null, as it's not important for us.

@CyrusNajmabadi  @cartermp  And I presume GetHelpTerm is just the thing returned by this code below?
    /// <summary>Compute the Visual Studio F1-help key identifier for the given location, based on name resolution results</summary>
    ///
    /// <param name="line">The line number where the information is being requested.</param>
    /// <param name="colAtEndOfNames">The column number at the end of the identifiers where the information is being requested.</param>
    /// <param name="lineText">The text of the line where the information is being requested.</param>
    /// <param name="names">The identifiers at the location where the information is being requested.</param>
    member GetF1KeywordAlternate                   : line:int * colAtEndOfNames:int * lineText:string * names:string list -> Async<string option>

Which was previously called in this code:
        override this.GetF1KeywordString(span : TextSpan, context : IVsUserContext) : unit =
            let shouldTryToFindIdentToTheLeft (token : FSharpTokenInfo) =
                match token.CharClass with
                | FSharpTokenCharKind.WhiteSpace -> true
                | FSharpTokenCharKind.Delimiter -> true
                | _ -> false

            let keyword =
                let line = span.iStartLine
                let lineText = VsTextLines.LineText (VsTextView.Buffer view) line
                let tokenInformation, col =
                    let col =
                        if span.iStartIndex = lineText.Length && span.iStartIndex > 0 then
                            // if we are at the end of the line, we always step back one character
                            span.iStartIndex - 1
                        else
                            span.iStartIndex
                    let textColorState = VsTextLines.TextColorState (VsTextView.Buffer view)
                    match colorizer.Value.GetTokenInformationAt(textColorState,line,col) with
                    | Some token as original when col > 0 && shouldTryToFindIdentToTheLeft token ->
                        // try to step back one char
                        match colorizer.Value.GetTokenInformationAt(textColorState,line,col-1) with
                        | Some token as newInfo when token.CharClass <> FSharpTokenCharKind.WhiteSpace -> newInfo, col - 1
                        |   _ -> original, col
                    | otherwise -> otherwise, col

                match tokenInformation with
                |   None -> None
                |   Some token ->
                        match token.CharClass, token.ColorClass with
                        |   FSharpTokenCharKind.Keyword, _
                        |   FSharpTokenCharKind.Operator, _
                        |   _, FSharpTokenColorKind.PreprocessorKeyword ->
                                lineText.Substring(token.LeftColumn, token.RightColumn - token.LeftColumn + 1) + "_FS" |> Some

                        |   (FSharpTokenCharKind.Comment|FSharpTokenCharKind.LineComment), _ -> Some "comment_FS"

                        |   FSharpTokenCharKind.Identifier, _ ->
                                try
                                    let lineText = VsTextLines.LineText (VsTextView.Buffer view) line
                                    let possibleIdentifier = QuickParse.GetCompleteIdentifierIsland false lineText col
                                    match possibleIdentifier with
                                    |   None -> None // no help keyword
                                    |   Some(s,colAtEndOfNames, _) ->
                                            if typedResults.HasFullTypeCheckInfo then
                                                let qualId = PrettyNaming.GetLongNameFromString s
                                                match typedResults.GetF1KeywordAlternate(Range.Line.fromZ line,colAtEndOfNames, lineText, qualId) |> Async.RunSynchronously with
                                                | Some s -> Some s
                                                | None -> None
                                            else None
                                with e ->
                                    Assert.Exception (e)
                                    reraise()
                        | _ -> None
            match keyword with
            |   Some f1Keyword ->
                    context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "devlang", "fsharp") |> ignore
                    // TargetFrameworkMoniker is not set for files that are not part of project (scripts and orphan fs files)
                    if not (String.IsNullOrEmpty projectSite.TargetFrameworkMoniker) then
                        context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "TargetFrameworkMoniker", projectSite.TargetFrameworkMoniker) |> ignore
                    context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_LookupF1_CaseSensitive, "keyword", f1Keyword) |> ignore
                    ()
            |   None -> ()

which was in turn used by this code:
        internal virtual int UpdateLanguageContext(LanguageContextHint hint, IVsTextLines buffer, TextSpan[] ptsSelection, IVsUserContext context)
        {
            // From the docs: Any failure code: means the implementer is "passing" on this opportunity to provide context and the text editor will fall back to other mechanisms.
            if (ptsSelection == null || ptsSelection.Length != 1) return NativeMethods.E_FAIL;
            context.RemoveAttribute(null, null);
            TextSpan span = ptsSelection[0];
            IVsTextLines lastActiveBuffer;
            IVsTextView lastAciveView = this.LastActiveTextView;
            if (lastActiveView == null) return NativeMethods.E_FAIL;
            NativeMethods.ThrowOnFailure(lastActiveView.GetBuffer(out lastActiveBuffer));
            if (lastActiveBuffer != buffer) return NativeMethods.E_FAIL;
            ISource source = GetSource(buffer);
            if (source == null) return NativeMethods.E_FAIL;

            var req = source.BeginBackgroundRequest(span.iStartLine, span.iStartIndex, new TokenInfo(), BackgroundRequestReason.FullTypeCheck, lastActiveView, RequireFreshResults.Yes, new BackgroundRequestResultHandler(this.HandleUpdateLanguageContextResponse));

            if (req == null || req.Result == null) return NativeMethods.E_FAIL;

            if ((req.IsSynchronous ||
                    ((req.Result != null) && req.Result.TryWaitForBackgroundRequestCompletion(1000))))
            {
                if (req.IsAborted) return NativeMethods.E_FAIL;
                if (req.ResultIntellisenseInfo != null)
                {
                    req.ResultIntellisenseInfo.GetF1KeywordString(span, context);
                    return NativeMethods.S_OK;
                }
            }
            else // result is asynchronous and have not completed within 1000 ms
            {
                context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "devlang", "fsharp");
                context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_LookupF1_CaseSensitive, "keyword", "fsharp.typechecking.incomplete");
                return NativeMethods.S_OK;
            }
            return NativeMethods.E_FAIL;

        }
        internal void HandleUpdateLanguageContextResponse(BackgroundRequest req)
        {
        }

@dsyme Yep, that's correct.

That interface is marked internal.

@rojepp that's not a problem, everything there is internal and we use it. I've no idea how it works though.

@vasily-kirichenko How do you get it to compile? I don't know much about Roslyn I'm afraid.

You cannot compile it? Strange. Make a WIP PR, we'll look at it.

It seems I can. I was doing something wrong. I'd be happy to do some work on this if no-one else is working on it?

What about me, I'm not going to work on it.

@rojepp pro tip: run build.cmd vs, then install VisualFSharpOpenSource.vsix before using VS 2017, it will make your life a lot easier :)

To get the latest fixes? Thanks, I'll do that!

Yes. Without it even tooltips don't work.

@rojepp that's not a problem, everything there is internal and we use it. I've no idea how it works though

FSharp.Editor is listed as InternalsVisibleTo for Roslyn.

BTW I've pushed a branch "bleeding-edge" which you can use to get the combination of features @vasily-kirichenko has been working on. PRs should of course be based on master.
https://github.com/Microsoft/visualfsharp/tree/bleeding-edge

@dsyme I'm afraid I've stuck with Inline Rename unless somebody from Roslyn/TS team helps.

@vasily-kirichenko Ah yes :) Bleeding indeed :)

@rojepp I just have some basic code that implements the interface, but it doesn't seem to be intercepting the f1 key hit.  Have you had success getting it to hit a breakpoint?  I noticed that the old language service implementation isn't even called anymore, so that isn't overriding this.
namespace Microsoft.VisualStudio.FSharp.Editor

open System
open System.Composition
open System.Collections.Generic
open System.Collections.Immutable
open System.Linq
open System.Threading
open System.Threading.Tasks
open System.Runtime.CompilerServices

open Microsoft.CodeAnalysis
open Microsoft.CodeAnalysis.Classification
open Microsoft.CodeAnalysis.Editor
open Microsoft.CodeAnalysis.Editor.Host
open Microsoft.CodeAnalysis.Navigation
open Microsoft.CodeAnalysis.Editor.Shared.Utilities
open Microsoft.CodeAnalysis.Host.Mef
open Microsoft.CodeAnalysis.Text
open Microsoft.VisualStudio.LanguageServices.Implementation.F1Help

open Microsoft.VisualStudio.FSharp.LanguageService
open Microsoft.VisualStudio.Text
open Microsoft.VisualStudio.Text.Tagging

open Microsoft.FSharp.Compiler.Range
open Microsoft.FSharp.Compiler.SourceCodeServices

[<Shared>]
[<ExportLanguageService(typeof<IHelpContextService>, FSharpCommonConstants.FSharpLanguageName)>]
type internal FSharpF1HelpProvider =

    static member internal GetKeywordAsync(document: Document, textSpan: TextSpan, cancellationToken: CancellationToken) =
        async {
            document |> ignore
            textSpan |> ignore
            cancellationToken |> ignore
            return Some("hello")
        }

    interface IHelpContextService with
        member this.GetHelpTermAsync(document: Document, textSpan: TextSpan, cancellationToken: CancellationToken) =
            async {
                let! res = FSharpF1HelpProvider.GetKeywordAsync(document, textSpan, cancellationToken)
                match res with
                | Some keyword -> return keyword
                | None -> return String.Empty
            } |> CommonRoslynHelpers.StartAsyncAsTask cancellationToken

        // The current interface has this member which is not applicable to F#.
        // We can just return 'null' here, as it won't affect the behavior for us.
        member this.FormatSymbol(symbol: ISymbol) = symbol |> ignore; null

        // This is what C# does.  See here:
        //
        // https://github.com/dotnet/roslyn/blob/84bc7eea2e05e9d9d7e1b53c2a107c0ac9857a72/src/VisualStudio/CSharp/Impl/LanguageService/CSharpHelpContextService.cs#L23
        member this.Language = "fsharp"
        member this.Product = "fsharp"

@cartermp I'm having the same issue. My code isn't getting called. I also tried inheriting from AbstractHelpContextService instead of just implementing the interface, like C# and VB does, but it didn't help. Maybe @CyrusNajmabadi could offer some insight? Is there some additional voodoo needed other than
[<Shared>]
[<ExportLanguageService(typeof<IHelpContextService>, FSharpCommonConstants.FSharpLanguageName)>]

?

Check (twice) that Shared attribute is the right one.

definitely don't derive from the Abtract guys.  They're there to share VB/C# logic.

You'll have to debug through AbstractLanguageService`2.UpdateLanguageContext
You guys do have a derivation of AbstractLanguageService`2 in F# land, right?

Okay, thanks @CyrusNajmabadi for pointing me in the right direction.
In our subclass of AbstractLanguageService`2 here, we'll need to do something similar to this, which will get the IHelpContextService service, which we implement as above.

no no... you shouldn't need ot do that.  we already do that for you :)

We need to figure out why it's not working.  can you add a BP to that method and see what happens when F1 is hit in an F# scenario?

@CyrusNajmabadi I don't have Roslyn set up for debugging, but I reimplemented UpdateLanguageContext in our FSharpLanguageService and but a BP in there. The call to document.Project.LanguageServices.GetService<IHelpContextService>() returns null.
    interface IVsLanguageContextProvider with
        member this.UpdateLanguageContext(_dwHint, pBuffer, ptsSelection, pUC) =
            let textViewAdapter = this.Package.ComponentModel.GetService<IVsEditorAdaptersFactoryService>()
            let textBuffer = textViewAdapter.GetDataBuffer(pBuffer)
            let context =  pUC :?> IVsUserContext

            match textBuffer = null, context = null with
            | true,_
            | _,true -> VSConstants.E_UNEXPECTED;
            | _ ->

            let snapshot = textBuffer.CurrentSnapshot
            match snapshot.GetOpenDocumentInCurrentContextWithChanges() with
            | null -> VSConstants.E_FAIL
            | document ->
            match document.Project.LanguageServices.GetService<IHelpContextService>() with
            | null -> VSConstants.E_NOTIMPL
            | helpService ->

            let selectionStart = textBuffer.CurrentSnapshot.GetLineFromLineNumber(ptsSelection.[0].iStartLine).Start + ptsSelection.[0].iStartIndex
            let selectionEnd = textBuffer.CurrentSnapshot.GetLineFromLineNumber(ptsSelection.[0].iEndLine).Start + ptsSelection.[0].iEndIndex
            let _span = Microsoft.CodeAnalysis.Text.TextSpan.FromBounds(unbox selectionStart, unbox selectionEnd)

            // VS help is not cancellable.
            let _cancellationToken = System.Threading.CancellationToken.None;
            let helpTerm = "DummyKeyword" // helpService.GetHelpTermAsync(document, span, cancellationToken).WaitAndGetResult(cancellationToken);

            match String.IsNullOrWhiteSpace(helpTerm) with
            | true -> VSConstants.S_FALSE
            | _  ->
            context.RemoveAttribute("keyword", null) |> ignore
            context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "devlang", helpService.Language) |> ignore
            context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "product", helpService.Product) |> ignore
            context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_Filter, "product", "VS") |> ignore
            context.AddAttribute(VSUSERCONTEXTATTRIBUTEUSAGE.VSUC_Usage_LookupF1_CaseSensitive, "keyword", helpTerm) |> ignore

            VSConstants.S_OK

Is the document correct?

It's the right document. It also has document.Project.Language = 'F#'.
I am confused whether IHelpContextService.Language and IHelpContextService.Product should be 'F#' or 'fsharp', though none of them seem to make a difference.

We'll have to debug to figure out why document.Project.LanguageServices.GetService<IHelpContextService>() is returning null.  I don't know enough about F# to know what's going on.  But it seems like your service isn't getting exported properly.

I am confused whether IHelpContextService.Language and IHelpContextService.Product should be 'F#' or 'fsharp', though none of them seem to make a difference.

Those shouldn't matter here, since we're not even getting to that point.  That's something to do with how the help system works.  Honestly, if you weren't setting those values before, they likely don't matter at all now.

@vasily-kirichenko System.Composition.SharedAttribute.
@CyrusNajmabadi Thanks for clarifying that

@OmarTawfik Is there anything special we might need to do to export IHelpContextService aside from having it defined in FSharp.Editor and decorated with [<ExportLanguageService(typeof<IHelpContextService>, FSharpCommonConstants.FSharpLanguageName)>]?  That's what other services do, but for some reason this one isn't getting picked up.

@cartermp first thing I noticed is that you should use FSharpCommonConstants.FSharpLanguageName instead of fsharp. Not sure where the this.Language is used in Roslyn.
Next step, if you cannot get it to be picked up, look for GetService<IHelpContextService>() in Roslyn code base. Put a break point there and you'll find one of two things:

The service is being MEF imported, but not used for some reason: easy fix, the Roslyn code should point out why it is not picked up.
The service is not being MEF imported: bigger MEF problem, and you'd probably need to use MEF-fu there.

@OmarTawfik I used FSharpCommonConstants.FSharpLanguageName in the attribute. this.Language is used to send the correct language attribute to MSDN. In previous VS, we added these ourselves here: https://github.com/Microsoft/visualfsharp/blob/0988f1fe19b4136ccf3e54d4fb78cefdb36e001f/vsintegration/src/FSharp.LanguageService/Intellisense.fs#L538

OK, problem solved at my end! I had open statements for System.Composition and System.ComponentModel.Composition. I removed the latter, and Roslyn picked up the new service. I'll continue working on this tomorrow. Thanks for the input!

@rojepp Awesome!  Looking forward to seeing this working.
Interesting that removing that open statement did the trick.

Great!

This is the exact problem I've met twice. I was sure it's about the wrong Shared attribute.

Seems like the kind of thing that a compile error or warning should be generated from.

Fixed by #1966.


Possible to customize the trigger keyboard command?

I don't want to have to type / to trigger the suggestion panel, I want to just start typing and have it read from my path mappings.
For example, in Liquid I would have this:
"path-autocomplete.pathMappings": {
    "/": "${folder}/snippets"
}

{% render '' %}

when my cursor is between the two quotes, I would like the panel to popup with my snippets list where I can just start typing the name of a snippet to auto-fill, without the preceding /. Is this currently possible?

You should get completions if you press Ctrl + Space
You will get completions from all the sources, so you may need to look at the bottom of the list to find the files from that folder.
You may also need to configure the $root property in the path mappings.
"path-autocomplete.pathMappings": {
    "$root": "${folder}/snippets"
}


SIL: Workaround for GenericSignatureBuilder bug [5.6]

The GSB will drop same-type requirements sometimes, when the
right hand side is smaller than the left. Change the order
when adding these requirements, which seems to work well
enough for my reduced testcase.
Fixes rdar://86431977.

@swift-ci Please test


Example notation does not work if the class is not in the same project.

Hi,
In my solution, I have separated the API and the model into two different projects, the problem is that the  tag is not recognized. The tag is recognized when the model is placed in the same project.
Steps to reproduce:
1: Upload a working solution.
2: Create a child project of type DLL and reference it to the API project
3: Move one of the classes returned by your api to the child project and update the references.
4: Recompile and run

I just found the way to include other project's coments. So im closing this issue...
Im using .NET6.0 so you might need to change something....
Thats my original Configuration code in AddSwaggerGen:
               var filePath = Path.Combine(System.AppContext.BaseDirectory, $"{Assembly.GetExecutingAssembly().GetName().Name}.xml");

New Code:
                var filePath = Path.Combine(System.AppContext.BaseDirectory, $"{Assembly.GetExecutingAssembly().GetName().Name}.xml");
                c.IncludeXmlComments(filePath);
                filePath = Path.Combine(System.AppContext.BaseDirectory, $"SecondaryProject.xml");
                c.IncludeXmlComments(filePath);

Note: Ensure that your "SecondaryProject"  is configured to generate the XML documentation file when compiles.


Switching from 1.5.0 to 1.6.0 (or later) causes exception on app startup

I've just updated WinUIEx from 1.5.0 to 1.8.0 but my application crashes on startup and this is what I get. I tried to downgrade WinUIEx from 1.5.0 to 1.7.0 and 1.6.0 and it didn't change.

I tried to downgrade WinUIEx from 1.5.0 to 1.7.0 and 1.6.0 and it didn't change.

Are you saying if you downgrade to 1.5.0 though, the problem goes away, indicating the issue started in 1.6.0?

Yes. If I create a new project using 1.8.0 directly it works, but in my existing project I'm stuck to 1.5.0: updating WinUIEx to 1.6.0 or later, I get this result.

Sorry for the slow response, but been pretty busy with life stuff. Any chance you could provide a simple sample that reproduces the problem? Sounds like a new project doesn't reproduce it. Otherwise I'd just be guessing what the problem is, and I'm terrible at guessing.

Closing. If you have more info feel free to reopen


gefyra up --minikube not working when Minikube was created with a defined profile

What happened?
I started a Minikube cluster with the following parameters:
minikube start -p beiboot --cpus=max --memory=4000 --driver=docker --addons=default-storageclass storage-provisioner

Please mind the -p beiboot argument which creates a dedicated cluster for this particular profile. When up-ing Gefyra, it says:
> gefyra up --minikube
[CRITICAL] There was an error running Gefyra: Could not find the Minikube configuration at ~/.minikube/profiles/minikube/config.json. Did you start Minikube?

What did you expect to happen?
Well, I would expect Gefyra to tell me that no cluster is running for the default profile. However, it does not consider multiple profiles. In addition, there is no option to specify the Minikube profile I want to connect to.
How can we reproduce it (as minimally and precisely as possible)?
Please see above.
What Kubernetes setup are you working with?

$ kubectl version
# paste output here

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Anything else we need to know?
No response

I just switched to the default profile with Minikube. Up-ing Gefyra was successful, but status reports no proper state:
> gefyra status
{
  "summary": "Gefyra is not running properly",
  "cluster": {
    "connected": true,
    "operator": true,
    "operator_image": "quay.io/gefyra/operator:0.13.4",
    "stowaway": true,
    "stowaway_image": "quay.io/gefyra/stowaway:0.13.4",
    "namespace": true
  },
  "client": {
    "version": "0.13.4",
    "cargo": true,
    "cargo_image": "gefyra-cargo:20221214183611",
    "network": false,
    "connection": false,
    "containers": 0,
    "bridges": 0,
    "kubeconfig": "~/.kube/config",
    "context": "minikube",
    "cargo_endpoint": "<IP_ADDRESS>:31820"
  }
}

Pinging Stowaway from the _gefyra-cargo _ (ping <IP_ADDRESS>) is working tho.

It seems there is a regression in Gefyra version 1.0.0. With this version the --minikube flag is not working at all. I rolled back to 0.13.4 and I can confirm it working again.


Implement support for Null values in columns

Implements support for null values in columns.
Updated the unit tests to include null values and added some new ones to test the more corner cases

using System;

copyright

Refers to: tests/Microsoft.Data.DataFrame.Tests/BufferTests.cs:1 in 9ce1ada. [](commit_id = 9ce1ada5149427c535e2d8b80aab01784efa747c, deletion_comment = False)

Maybe a good test to have would be that you can take an Apache.Arrow.RecordBatch and wrap a DataFrame around it without any copies. I'm wondering if the "null" support you are adding is compatible with the Arrow "null" format.

Oh also, how do folks reply to comments like this on GitHub? I don't see a "Reply" option, just an "Edit" option

Typically, you reply to comments like this on GitHub at the bottom, and using a > quote of the comment you are replying to.

Thanks for the review @gfoidl  and @eerhardt


game director needs write access to it's code directory

it errors without that due to failed logging.
Also the config.py file must be edited to change the interfaces.
The documentation has instructions how to change permissions.

Emailed request to Chuck.

Resolved upstream


Add a :post/sites attribute to the :post schema

Follows the recommended minimal implementation:
use a native html single-select element
Resolves #39

Looks great; Github says there are merge conflicts though.

Yeah, passing the db felt awkward; thanks for the tip!

Huh that's strange--github shows me this:

Must be a glitch, I see your PR is already based on the latest master commit anyway. I'll just figure out how to merge it from the CLI :).

I used the same fork as my previous PR, doing a git pull to be sure I didn't miss something.
This resulted in the merge commit and fork's previous commits looking like new commits in the subsequent PR.  That may have caused this situation.


Can now add footer from the designer

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Please also include any acceptance criteria if you have any.

Add a section to the designer for adding footer data to the form.

This leads to a new section for adding the new footers to the form data

Once saved this adds to the form data

And displays on the page when previewing

Type of change
Please delete options that are not relevant.

[ ] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce
the testing if necessary.
Before PR's can be merged they will need to be tested by QA and approved where
applicable. To flag the change to QA assign @XGovFormBuilder/qa as one of the reviewers.

[ ] Create a new form
[ ] Click on the new footer menu item
[ ] Add a new footer
[ ] Delete a footer
[ ] Change a footer
[ ] View the json data on the summary page
[ ]  Click preview to see the footer

Checklist:

[ ] My changes do not introduce any new linting errors
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I have rebased onto main and there are no code conflicts
[ ] I have checked deployments are working in all environments

Is this still relevant @adamdavies1 - looking through all opens prs

Is this still relevant @adamdavies1 - looking through all opens prs

@tferns So its not a required change, its a change i was working on in my spare time to start to get the designer up to date. Lets hold fire for now until we get all the rest done


Calculate downregulated genes?

Hi,
is there a possibility to calculate downregulated genes between two clusters? In the function tl.rank_genes_groups() I did not find such option though it should be possible with the Wilcoxon test.
Afaik in Seurat.FindMarkers() there is an option only.pos for the Wilcoxon test (https://www.rdocumentation.org/packages/Seurat/versions/3.0.0/topics/FindMarkers).
Following another discussion here about DEG I tried to switch to MAST to get around that but it seems to be available only through R (https://github.com/RGLab/MAST/issues/102). Also Wilcoxon did reasonable well in a recent paper (https://www.nature.com/articles/nmeth.4612)..
Thanks!
Tilo

You can interchange the cluster order (reference with the cluster your intersted in) and get only-upregulated ones for the inverted comparison (hence down-regulated genes) or set rankby_abs to True, which will then give you down-regulated genes in addition to up-regulated ones.

rankby_abs=True - that was easy. Thanks

In case you're still looking to use MAST or integrate other R tools into a scanpy pipeline. That works quite well via anndata2ri. An example of how you can do this can be found in the case study notebook here.

That's a great workaround!


7.8.0 Release

This PR prepare 7.8 branch for 7.8.0 release:

bump chart and Docker images version from 7.8.0-SNAPSHOT to 7.8.0
remove notices on README that this is a development branch and development branch specific install instructions

Note

remove notices on README that this is a development branch and development branch specific install instructions

This is required because released Helm charts will be packaged from this PR commit including the README files that will be available inside the chart packages and in Helm Hub pages (example: Elasticsearch chart page).
This change will be reverted in the next PR after release when we will bump 7.8 branch to 7.8.1-SNAPSHOT version.
Replace #672

7.8.0 Release changes (to add to GitHub release and changelog):
7.8.0 - 2020/06/18

7.8.0 as the default stack version
Helm 2.16.9 support in #667 @jmlrt

APM Server

PR
Author
Title

#664
@jmlrt
Increase memory limit

#671
@jmlrt
Allow customizing probes

Elasticsearch

PR
Author
Title

#437
desaintmartin
Remove heritage from selector

#542
@floretan
Disable service links to prevent very long startup times

#671
@jmlrt
Update kind example for version >= 0.7.0

Logstash

PR
Author
Title

#392
@jmlrt
Add security example

elastic+helm-charts+pull-request+integration-logstash/287/KUBERNETES_VERSION=1.15,LOGSTASH_SUITE=default is a transient error not related to this PR.
The other tests are OK