Go Daddy Hosting

First, thank you for both the wonderful code and the great examples.  I'm not much of a javascript programmer, but I was able to start with your examples and, with a little reference on MapBox, get the maps behaving EXACTLY the way I wanted them to.  On Localhost.
When I moved my code up to my GoDaddy, it didn't work.  I can access the page, and a WMS basemap works fine.  But nothing that depends on mbtiles.  I've checked, and both pdo_sqlite and GD extensions are installed.  Any other thoughts?  Is there a minimum php release?  My account was running php 5.3.24, I believe.  It's definitely running Apache.

Hi! I have no clue if this works with GoDaddy. Your PHP version is fine, and Apache definitely works. My first hunch is that it's probably something with your .htaccess file, but that's hard to say unless I can investigate a bit... Do you have a link?

Hey Edward!  Thanks for the quick response.  Go Daddy is now running PHP5.4.19 (something I changed as part of debugging, before I decided to contact you).
Anyway, thank you for your help.
I've put up two versions of your page on my site:
http://wildtroutstreams.com/example/index.php  (this includes phpinfo();)
http://wildtroutstreams.com/example/index-layers2.html
Both pages serve up a file: pa_test_2.mbtiles
Both work more or less the way I want them to on Localhost.
If you look at the layers2 page, you'll see a leaflet map of PA served from a National Map WMS server.  You're supposed to be able to click on the layers control, and select PA Trout, to overlay the tiles which show streams holding wild trout.  Nothing happens.  Similarly, the index.php page throws up an empty div with just the zoom control.

Hi! No problem. When I visit either of those pages, I see that it tries to load the map, and it is doing that at  the http://wildtroutstreams.com/example/server/ address (specifically, it tries to request http://wildtroutstreams.com/example/server/pa_test_2.tilejson). When I visit either of those links, I get a 404 "Category not found" error. Are you sure that you copied all the files to the server?
You need to have 'server' folder under the 'example' folder. It looks like that folder does not exist, because I can't access any of these: http://wildtroutstreams.com/example/server/, the http://wildtroutstreams.com/example/server/pa_test_2.tilejson or even the http://wildtroutstreams.com/example/server/.htaccess file.
To me that looks like the folder isn't there. Usually if the folder is there, and you try to access the .htaccess file, you don't get a 404 error but a 'permission denied' or 'server misconfigured' error. I now get a 404 'file not found' error.
Can you check if you have uploaded everything?

I used FileZilla to upload everything.  It shows all of the files in /example/server, all with the same byte count.  I just added a random picture to the folder and it's definitely there:
http://wildtroutstreams.com/example/server/tostadas.jpg
Filezilla shows all of the permissions of the files within the server folder as 0604
It shows permissions of the server folder (when you're in /example) as 0705

Ah, I see. I think you haven't uploaded the .htaccess file then. Can you confirm that it's there?

Sorry, no, it's there.  192 bytes on my localhost and on GoDaddy

One thought... the root site is a joomla site that has a very elaborate .htaccess file for friendly urls.  Could the two be in conflict?

Yeah, they could be. I tried to access that file (the one in the root), but I couldn't access it. Can you post it here?

You mean the root .htaccess file?

Yes.

OK, I downloaded it...

@version		$Id: htaccess.txt 21101 2011-04-07 15:47:33Z dextercowley $
@package		Joomla
@copyright	Copyright (C) 2005 - 2011 Open Source Matters. All rights reserved.
@license		GNU General Public License version 2 or later; see LICENSE.txt

READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!

The line just below this section: 'Options +FollowSymLinks' may cause problems
with some server configurations.  It is required for use of mod_rewrite, but may already
be set by your server administrator in a way that dissallows changing it in
your .htaccess file.  If using it causes your server to error out, comment it out (add # to
beginning of line), reload your site in your browser and test your sef url's.  If they work,
it has been set by your server administrator and you do not need it set here.

Can be commented out if causes errors, see notes above.
Options +FollowSymLinks
Mod_rewrite in use.
RewriteEngine On
Begin - Rewrite rules to block out some common exploits.
If you experience problems on your site block out the operations listed below
This attempts to block the most common type of exploit attempts to Joomla!

Block out any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]([^)]) [OR]
Block out any script that includes a

sorry, I tried cutting and pasting but that seems like a mess.  How do I attach a .txt file?

I edited it :)

'##
@version		$Id: htaccess.txt 21101 2011-04-07 15:47:33Z dextercowley $
@package		Joomla
@copyright	Copyright (C) 2005 - 2011 Open Source Matters. All rights reserved.
@license		GNU General Public License version 2 or later; see LICENSE.txt

READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!

The line just below this section: 'Options +FollowSymLinks' may cause problems
with some server configurations.  It is required for use of mod_rewrite, but may already
be set by your server administrator in a way that dissallows changing it in
your .htaccess file.  If using it causes your server to error out, comment it out (add # to
beginning of line), reload your site in your browser and test your sef url's.  If they work,
it has been set by your server administrator and you do not need it set here.

Can be commented out if causes errors, see notes above.
Options +FollowSymLinks
Mod_rewrite in use.
RewriteEngine On
Begin - Rewrite rules to block out some common exploits.
If you experience problems on your site block out the operations listed below
This attempts to block the most common type of exploit attempts to Joomla!

Block out any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]([^)]) [OR]
Block out any script that includes a

Thanks, you got the whole thing?  I was madly scanning the Markdown info

Yes, I got it (see above). I edited it by putting three backticks ` before and after it. Now let me have a look...

Yeah, I'm pretty sure that the last part is interfering with the mbtiles-php. You can check if that's the case by temporarily renaming Joomla's .htaccess file and then checking if the map loads.
If that is the case, then you need to adjust Joomla's .htaccess file. I think you need to put something like this just after the RewriteEngine On statement:
RewriteRule ^(example)($|/) - [L]

This would/should stop Joomla with interfering with rewrite-rules that are defined in the 'example' directory, according to this StackOverflow post. But I haven't tested this!
Nevertheless, if the map loads if you temporarily rename Joomla's .htaccess file, then we have found the culprit ;-)

Unfortunately, I just satisfied myself that it's NOT the .htacess file.  I just installed the same directories in a different domain.  It's in the same hosting account, but on a domain that has nothing in the root directory.  Behavior is the same...
http://greendrakeservices.com/example/index-layers2.html

Hmm. That one is weird - it tries to redirect to the wrong URL. http://greendrakeservices.com/example/server/index.php wants to redirect to "/greendrake/greendrake/".."/tileserver.php" instead of just "/greendrake", and it can't find http://greendrakeservices.com/example/server/tileserver.php which should certainly be there.

I have no idea why...  I can probably take this to GoDaddy support and ask them.
You've undoubtedly looked at my code.  I just create a variable troutmbtiles that is a relative URL that should work in any context.

Yeah, I don't think your code is the problem here. It some sort of routing problem. The request never even gets to the PHP file, and that's why the map doesn't show. It could be that GoDaddy doesn't allow .htaccess files in subdirectories, or something inane like that. If that's the case, you can try this before you contact them:
Options -Indexes
Options +FollowSymLinks
Options -MultiViews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ example/server/tileserver.php [QSA,L]

Put that in a new .htaccess file in the root of the greendrakeservices server and see if it loads then.

Just added your .htaccess file in the root of green drake services, and the final behavior is no different.  Does it change the URL it requests?  In any event, do you have any words of wisdom for how I should describe the problem to GoDaddy?
My inclination would be to erase the root .htaccess file you wrote, and ask them why greendrakeservices isn't routing correctly using the /example/server .htaccess file (since the Joomla .htaccess seems to be a red-herring).  I did look up usage of .htaccess in GoDaddy's support pages, and they simply say they work.  I also think that apps like WordPress wouldn't work if they restricted .htaccess to the root directory.

I think your approach makes sense. Just delete the .htaccess file I wrote and ask them why it doesn't route correctly. Sorry I couldn't help you - it's difficult to fix such a problem remotely without access to the system. Good luck contacting GoDaddy! I hope you get it to work.

Thanks for your help!  I'll post what I find out.
One thought... the name of the root directory for greendrakeservices.com in my hosting account is "greendrake".  So it looks like its rewriting the relative url to something that includes the name of the root directory (which I assume should instead simply be '/' within the context of the web site).
Can you just tell me exactly what it's rewriting to?  (i.e. filling in the /.../)  As well, what it SHOULD be rewriting to?  I have a pretty good idea, but I might be wrong.  I've deleted the root .htaccess file.
BTW, before when I added the root .htaccess file, I didn't delete the the one in the /server folder.  Should I have done that?
This is important enough that I would change hosts to make it work.  WildTroutStreams publishes geographic info (e.g. KMLs) about where to find wild trout in 38 states. The need to download the data into something like Google Earth is a major barrier to a lot of my potential audience. So in the next generation of the site, I'd like to publish the data in web maps with the option to download the KMLs.
But I have dozens of maps (at least 38, plus many other maps for Native trout species), and several GB of data.  I intentionally have no business model for the site (everything's free).  Any single layer in a single state  can be up to about 20 MB of vector features in a KML, and a map can have anywhere from one to half a dozen layers.  So the conventional web hosting models would cost a fortune.
Your solution is a godsend, and seems ideal for my needs which is a small user base, and lots of maps, and tons of data.

Hmm, maybe you can try adding a RewriteBase / or something like that to the .htaccess file. Unfortunately .htaccess files can be tricky, and mod_rewrite rules can get pretty complicated and opaque. On top of that, if you make one tiny mistake it tends to explode.
The way the .htaccess is set up is that it should redirect all incoming requests to the tileserver.php file that resides in the 'server' directory. Since the .htaccess file itself lives in the 'server' directory, it should only look at requests that try to access something that is supposedly located in the 'server' directory. It does (and should not) operate on other requests.
The tileserver has an internal routing mechanism that looks at the request and then either sends a tilejson file, a utfgrid file, or a map tile – all extracted from the .mbtiles file or generated as necessary. I do think that the most likely outcome is that it is possible to host it on GoDaddy, but that we just haven't found the correct magic incantation yet.
If you want to I can have a look at it, but then you'd have to give me the username and password (and the login page) of your godaddy account. My email is in my profile page, but I can't look at it until tomorrow and I won't spend more than 15 minutes on it ;-)

First, thank you for the time you've already spent.  If you could take a look at it just so I can ask an accurate question of GoDaddy, that would be great.
Do you need anything other than ftp access?  I can readily set you up with an ftp account that gives you access to the example directory on greendrakeservices and nothing else.

PS, I did try the RewriteBase / suggestion and it didn't work.
I also tried this on another shared account on Network solutions that I use for a client, and had the same symptoms.  So it may be a generic issue with the way shared hosts are configured.

No problem. I think the question you are going to ask GoDaddy was fine. It gives them enough information and it has a nice isolated test case, so they should know what to do.
If you do give me access, then I'm not sure if just access to the 'example' directory would be enough.

Sigh.  GoDaddy tech support (at least via their chat interface) was completely useless.  Couldn't point me to any useful resources, and refused to look at the .htaccess code (which is considered providing programming support and beyond their remit).
Their only useful feedback was a suggestion to turn on the error logging in the hosting account.  It takes 24 hours to come online, so I can't see the error logs yet, but hopefully will do so soon.
I've been scouring the Internet for advice on .htaccess and GoDaddy.  It's frustrating because there's a ton of examples out there, but it's hard to know what's applicable or not.  I've probably tested a dozen or more different .htaccess configurations, based on suggestions I've found, none of which has solved the problem.
One of the more interesting pieces I found is this:  http://forum.ait-pro.com/forums/topic/go-daddy-redirect-htaccess-code-no-longer-working/
It's specifically about redirects (which aren't in your .htaccess file) but I wonder if there's a similar issue with rewrites.  All GoDaddy says is that mod_rewrite is turned on in all of their accounts and should follow standard apache coding practice.  Which is clearly bs.  Might it relate to our problem?  It resonated with the fact that you were finding the name of my host folder in the URLs that timeserver.php was seeing.
I'm very frustrated because this is an important application to get working.  I'm also talking to DreamHost about opening a VPS hosting account with them.  Presumably, I'd have enough control over the environment in that case to get this working.  Do you have experience with any shared hosting providers where your stuff just works?

Hey! Well, any VPS would work, because you have complete control over what it does and doesn't do. I'm not on shared hosting providers anymore for exactly these reasons (lack of control). But if you setup a VPS you probably need to do all the server administration yourself, which is not that easy and it's difficult to get right and keep secure. Just have a look at this to get a grasp of what you might need to concern yourself with.
Regardless, I do think it makes sense to try to get it to work with GoDaddy first. Setting up and maintaining a VPS can be quite a hassle. Did you try installing it in the root of the greendrake server? Did that at least work? Alternatively, do have a look at the logs once they come online...

When you say "installing it on the green drake server" do you mean putting index-layers2.html in the root, and keeping /server?  Or do you mean everything in the root, getting rid of the /server directory?
With respect to getting it to work on GoDaddy first, I'd love to do that, but I'm out of ideas. I now have access to the error logs, though there seems to be a delay in updating them.  Perhaps when I see some relevant entries it will stimulate something.

Yeah, I meant putting the whole contents of the 'example' directory in the root folder, including .htaccess and server folder. Curious if that works.

If it doesn't, maybe you can try removing the 'Options' lines in the .htaccess file. Perhaps GoDaddy doesn't like those...

OK, so I combined into one folder the .htaccess, the mbtiles file, tileserver.php, and index-layers2.html.
I also changed the relative address in index-layers2.html so it asks simply for the mbtiles file, without a relative subdirectory.  Presumably, if the problem is the rewrite base, this should eliminate it.
No joy.  You can see the setup at http://greendrakeservices.com/example2/index-layers2.html
Updating the error logs seems to be about 2 hours behind, so still nothing useful there.
The web is filled with people fixing their .htaccess files on GoDaddy by adding the same options you have, so I don't think that's the issue.  I need to go for a while.  When I get back I'll check the error logs and post anything interesting.  Also will try it without the options, just for giggles, but no time know.  Thank you for your support!

Well... I haven't given up by any means, though I've given up on GoDaddy.  I have opened up a VPS account on DreamHost.  I've got a subdomain now pointing at a directory on that server (maps.wildtroutstreams.com).  The html files are in the root directory for that subdomain, and timeserver.php and the .mbtiles file is in /server.
Dreamhost has a "standard" configuration option to go with nginx instead of apache, so I'm inclined to go in that direction anyway, since speed will be at a premium once users start using this.  I just wanted to confirm what I need to do to set the .conf file.
Dreamhost recommends setting up domain-specific .conf files to customize Nginx which goes somewhere like:
/home/myusername/nginx/maps.wildtroutstreams.com/tileserver.conf
Since this is the first time I'll be actually editing a file using a command line in 40 years (last on an IBM-360/67), here's what I intend to do (unless you suggest otherwise):

Create the directory:
mkdir -p /home/myusername/nginx/maps.wildtroutstreams.com/

Create the .conf file:

nano /home/myusername/nginx/maps.wildtroutstreams.com/tileserver.conf

Add your suggested code to the file using nano (which I'm going to have to figure out):

location /server/ {
try_files $uri /server/tileserver.php$is_args$args;
}
Does that seem about right?

Ok, that seems about right, but three remarks:

I don't know where your nginx site configuration files should be. Mine are in /etc/nginx/sites-available, and I have a symlink to each of those files in /etc/nginx/sites-enabled. Please make 100% sure that the location you mentioned is actually read by Nginx (you can test this by putting nonsense in it and then restarting Nginx. If it barks, you're good). If this is the location recommended by DreamHost then it's probably fine but it's better to be sure.

The mkdir and nano commands are fine

I looked up my own Nginx config for the maps to be sure if my original recommendation is still good. Turns out I'm using a different syntax - which I will post here for inspiration and reference:

    location ~ ^/infostreams\.net/.+/map/((?!tileserver).)*$ {
        rewrite ^/infostreams\.net/(.*)/map/(.*) /infostreams.net/$1/map/tileserver.php last;
    }

Basically this says that all requests to <site root>/infostreams.net/<something>/maps/ that do NOT mention tileserver.php should go directly to tileserver.php.
However, I do have to say that it does seem to me that your original rough patch (i.e. the maps not loading) might have been replaced by a larger rough patch (the configuration of a VPS). Good luck!

As the saying goes, I may have jumped out of the frying pan and into the fire with this VPS thing.  However, I don't trust my current GoDaddy hosting plan to provide good service anyway once I get some users onto this thin.  This, I hope, is a discrete task where I can get support.  In general, with the Dreamhost VPS plan, you can manage a lot of things from a control panel, just like a shared hosting plan.  I'm hopeful that I won't need to make many changes.
Question for you:  you have two levels of directories from the root.  Currently I'm doing only one: /server.  Programmatically, I can see some advantages to what you did (e.g. one for each state of the US), though I was planning on putting everything in one directory and just having a different .mbtiles file per state.
But, if i keep it two levels the way you do, would the code be:
'''location ~ ^/maps.wildtroutstreams.com/.+/server/((?!tileserver).)$ {
rewrite ^/maps.wildtroutstreams.com/(.)/server/(.*) /maps.wildtroutstreams.com/$1/server/tileserver.php last;
}'''
Conversely:  if i keep it one level, I just eliminate:
.+/ in line 1 and $1/ in line 2?

No, sorry - my example is a bit weird because the url contains the domain name AFTER the original domain name. I'm not going to share it here because I don't want google to pick up my unfinished new website yet, but it's something like http://subdomain.infostreams.topdomain/infostreams.net/projects/customer-name. The maps for that customer are hosted at http://subdomain.infostreams.topdomain/infostreams.net/projects/customer-name/server/.
So the 'infostreams.net' part that is listed in the Nginx rule is actually part of the PATH, not of the domain. In your case you'd have to tell Nginx to listen to maps.wildtroutstreams.com with something like
server {
    listen 80;
    server_name maps.wildtroutstreams.com;

    location ~ ^/server/((?!tileserver).)$ {
        rewrite ^/server/(.*) /server/tileserver.php last;
    }
}

Something along these lines at least...

Thanks...  also while you were answering my question I learned the difference between a back tic and a tic!  You've given me enough to take a shot at it.

Awesome. Well, good luck then!

Well, I'm back, having made some progress.  My VPS is configured with Nginx, and I have a .conf file that is being loaded and affecting how the server is rewriting URLs.  Unfortunately the code you provided isn't working.
Here's the content of tileserver.conf as it exists right now.  It gets loaded into the server section of the main nginx.conf file (and allows me to customize it without editing it directly).
        rewrite ^/server/(.*) /server/tileserver.php last;
    }
location /dhtest {
   rewrite ^/.* http://google.com/ permanent;
}```

The first 3 lines are your code.  The last 3 lines were added by Dreamhost tech support to demonstrate that the .conf file is being loaded.  If I enter http://maps.wildtroutstreams.com/dhtest I end up on google.com, which proves that the file is getting loaded.

Any thoughts?

Hey, Ed.  Sometimes we're smarter than we remember being.  Anyway, I couldn't get the new code you suggested in this thread to work in the configuration.  But I decided to try the code you provided in the original read me file instead (mostly out of desperation).  It turns out, THAT WORKS!!!  Yeah!
As I mentioned before, you can see the setup at http://greendrakeservices.com/example2/index-layers2.html.  I'll warn you that the current overlay looks like crap, but I can now check or uncheck the box and the overlay comes up.  The mapbox.js tooltips and popups also work.  So I think I'm off to the races in terms of building maps that I want to share with the world!
I may keep it up for a while, just so people who are looking to implement a similar application have it for reference.
But I'm now very excited and about the potential.  I'd spent some time trying to figure out how this could work, and your code is a godsend.  Thank you!  I'll let you know when I launch some real maps.

Hi! I'm really happy that you managed to figure it out :-) Hope you get your maps to work as envisioned. Happy hacking!

Can I close this issue?

Yes, you can, thank you!  It's still a work in progress, but I'm now working on data content not technology.  The performance (at least with minimal server load) is completely acceptable.  Check it out at:
http://maps.wildtroutstreams.com/PA.html

Awesome! Looking really good :+1:


Attempt using openai whisper instead of facebook wit.ai

https://github.com/openai/whisper
Installing this extra dependency will slow down my run pipeline, but may result in more accurate readings.

Advantages, will be more reliable and should be able to exit more frequently, without dropping as much frames.
Hopefully performs better than facebook, dall-e performance has been pretty good recently.
https://github.com/openai/whisper

closed by #32


Git compare functionality

I love atom's git integration... but it only marks that changes exist in folder/file/lines as compared to the git HEAD. I would like to implement a feature to compare any two branch/commit/tags, and eventually a tree visualization graph. Do either of these features make sense to be integrated in this repo? If not, are there any other projects/repo/packages that I should look into contributing to?

Like this?
https://github.com/jisaacks/GitGutter#comparing-against-different-commitsbranchestags

Possible duplicate of #74

@tuckbick, bingo that is exactly it, be able to change the comparison commit from HEAD to any branch/tag/commit
second feature request/question is perhaps clicking on the gutter to show the diff of the two files


Confusion in Code for ViewCustomerSummaryCommand.ts

[Enter feedback here]
can You verify,whether code will be
import MessageDialog from "../DialogSample/MessageDialog";
or
import MessageDialog from "../../DialogSample/MessageDialog";
in ViewCustomerSummaryCommand.ts Files

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cfef165f-aa45-f586-bffc-964ab04013fe
Version Independent ID: abc97ef9-6408-d006-dea5-ab49a3dd2da0
Content: Extend POS views to add custom columns and app bar buttons - Commerce | Dynamics 365
Content Source: articles/commerce/dev-itpro/pos-view-extension.md
Service: dynamics-365-retail
Product: ****
Technology: ****
GitHub Login: @mugunthanm
Microsoft Alias: mumani

Updated the sample code to:
"../../Controls/DialogSample/MessageDialog";
The full sample code is available in "\RetailSDK\POS\Extensions\SampleExtensions\ViewExtensions\Search\ViewCustomerSummaryCommand.ts"


Extend RunReactNativeXamlCodeGen target to support multiple WinMDs

Summary
As of PR #234 , there is a target named RunReactNativeXamlCodeGen which runs the codegen as part of the RNX native build.
Right now it only supports specifying one WinMD file, but the Codegen can actually support more, so we should allow users to specify more.
Motivation
If a user wants to make sure that Codegen is run, it's probably because they want to add multiple WinMDs from custom control libraries they want to add.
Basic Example
No response
Open Questions
No response

Current customers should be unblocked with the linked PR. Will wait to prioritize supporting multiple for when there's more ask for that use case.


stop checking customer pods storage utilisation

master: check all
infra: check all
node: don't check pod pv

👍


Unexpected behaviour adding domains when scope is only *.example.com

When starting with a new program and defining scope only with:
bbrf scope in add *.example.com
trying to add the domain example.com will fail.
For replying the behaviour try:
bbrf new example
bbrf use example
bbrf inscope add *.example.com
bbrf domain add example.com
bbrf domains
Output will be empty.

Hi,
Your bbrf server is up!?

Hi @rsilvav - this is expected behaviour. BBRF at the moment does not match example.com to *.example.com, as it doesn't consider it a strict subdomain.
So I would recommend adding example.com as in scope explicitly.
I have some ideas about changing the default behaviour, but nothing concretely planned.

Checking the matches_scope function i noticed that this specific flow avoids both literal and x.example.com matches. An user can avoid this behaviour by "forcing" the exact match adding always example.com to the scope, but in my opinion, this isn't intuitive.
An additional checking like the following could avoid this behaviour:
if s.startswith('*.'):
if domain.endswith('.'+s[2:]) or s[2:]== domain:
return True

@honoki thanks for your answer!
Strictly speaking you're right, but programs with wide scope (like *.example.com) usually are implicitely including example.com (maybe they are wrong?)
It is an ambiguous, but in my opinion is better to send some out-of-scope reports than missing some implicitely-valid bugs. Maybe inverting  the  behaviour (example.com matches *.example.com unless example.com is out-of-scope) is an alternative.
For now i will follow your advice :)


Init SpaceServer with existing Redis pool

This patch adds the option to build a SpaceapiServer by initializing it
with an existing r2d2_redis Pool.
I'm not too happy of that big match block, but if I use .unwrap_or_else it just goes down in unwrap hell... Any suggestions?
I still have to add tests, as soon as I understand more about testing with rust 😊
When merged, it should close #84 .

@g5pw Can you rebase against master? Some stuff from @dbrgn got merged in the meantime.

No problem, rebased!

Circle CI did not trigger because "Build forked pull requests" was set to off. I changed it to on. Can you do a push to retrigger?

I couldn't yet test it locally, so my review is only based on looking through the code on GitHub. @dbrgn do you want to take a look at it as well?

Would it make sense to add some unit testing?

Would it make sense to add some unit testing?

It's always better to have some unit tests, so if you feel like it go ahead and add them! But I think we can also merge this PR without them.

Ok, I think this is ready for merging :)

Right, I should have addressed all points now :)

@rnestler feel free to merge if you agree.


decide how to direct inquiries for projects & connections to TeachOSM

we don't want to be stuck as matchmaker, but want educators and data users to be able to network

[Issue here is: how do we scale as an organization to handle outside inquiries.]

Rendered moot since we have an email address + other ways of handling this.


HADOOP-19221. S3A: Unable to recover from failure of multipart block upload attempt (#6938)

This is #6938 for branch-3.4.1
This is a major change which handles 400 error responses when uploading large files from memory heap/buffer (or staging committer) and the remote S3 store returns a 500 response from a upload of a block in a multipart upload.
The SDK's own streaming code seems unable to fully replay the upload; at attempts to but then blocks and the S3 store returns a 400 response
"Your socket connection to the server was not read from or written to
 within the timeout period. Idle connections will be closed.
 (Service: S3, Status Code: 400...)"

There is an option to control whether or not the S3A client itself attempts to retry on a 50x error other than 503 throttling events (which are independently processed as before)
Option:  fs.s3a.retry.http.5xx.errors
Default: true
500 errors are very rare from standard AWS S3, which has a five nines SLA. It may be more common against S3 Express which has lower guarantees.
Third party stores have unknown guarantees, and the exception may indicate a bad server configuration. Consider setting fs.s3a.retry.http.5xx.errors to false when working with such stores.
Signification Code changes:
There is now a custom set of implementations of
software.amazon.awssdk.http.ContentStreamProvidercontent in the class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.
These:

Restart on failures
Do not copy buffers/byte buffers into new private byte arrays, so avoid exacerbating memory problems..

There new IOStatistics for specific http error codes -these are collected even when all recovery is performed within the SDK.
S3ABlockOutputStream has major changes, including handling of Thread.interrupt() on the main thread, which now triggers and briefly awaits cancellation of any ongoing uploads.
If the writing thread is interrupted in close(), it is mapped to an InterruptedIOException. Applications like Hive and Spark must catch these after cancelling a worker thread.
Contributed by Steve Loughran
How was this patch tested?
For code changes:

[X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

running local tests. this patch is a precursor to #7089, so if we want that in 3.4.1, this is needed first.

failure. looks (a) unrelated. (b) a race condition. (c) latent for a while
.s3a.ITestS3AContractVectoredRead
[ERROR] testSomeRangesMergedSomeUnmerged[Buffer type : array](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.905 s  <<< ERROR!
java.util.ConcurrentModificationException
        at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1728)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
        at org.apache.hadoop.fs.store.audit.HttpReferrerAuditHeader.buildHttpReferrer(HttpReferrerAuditHeader.java:182)
        at org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor$LoggingAuditSpan.modifyHttpRequest(LoggingAuditor.java:388)
        at org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A$WrappingAuditSpan.modifyHttpRequest(ActiveAuditManagerS3A.java:871)
        at org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A.modifyHttpRequest(ActiveAuditManagerS3A.java:612)
        at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyHttpRequestAndHttpContent(ExecutionInterceptorChain.java:89)
        at software.amazon.awssdk.core.internal.handler.BaseClientHandler.runModifyHttpRequestAndHttpContentInterceptors(BaseClientHandler.java:157)
        at software.amazon.awssdk.core.internal.handler.BaseClientHandler.finalizeSdkHttpFullRequest(BaseClientHandler.java:83)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:151)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:66)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:60)
        at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52)
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:60)
        at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:5174)
        at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:9005)
        at org.apache.hadoop.fs.s3a.S3AFileSystem$InputStreamCallbacksImpl.getObject(S3AFileSystem.java:1934)
        at org.apache.hadoop.fs.s3a.S3AInputStream.lambda$getS3Object$7(S3AInputStream.java:1223)
        at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
        at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
        at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
        at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
        at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
        at org.apache.hadoop.fs.s3a.S3AInputStream.getS3Object(S3AInputStream.java:1220)
        at org.apache.hadoop.fs.s3a.S3AInputStream.getS3ObjectInputStream(S3AInputStream.java:1117)
        at org.apache.hadoop.fs.s3a.S3AInputStream.readCombinedRangeAndUpdateChildren(S3AInputStream.java:963)
        at org.apache.hadoop.fs.s3a.S3AInputStream.lambda$readVectored$5(S3AInputStream.java:945)
        at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:225)
        at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:225)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

failure tracked in HADOOP-19299; unrelated. this is just the test run which found it.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 21 new or modified test files.

_ branch-3.4.1 Compile Tests _

+0 :ok:
mvndep
4m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
40m 56s

branch-3.4.1 passed

+1 :green_heart:
compile
18m 44s

branch-3.4.1 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
17m  2s

branch-3.4.1 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 27s

branch-3.4.1 passed

+1 :green_heart:
mvnsite
2m 40s

branch-3.4.1 passed

+1 :green_heart:
javadoc
1m 58s

branch-3.4.1 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 43s

branch-3.4.1 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
3m 57s

branch-3.4.1 passed

+1 :green_heart:
shadedclient
34m 38s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 34s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 31s

the patch passed

+1 :green_heart:
compile
17m  2s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
17m  2s

the patch passed

+1 :green_heart:
compile
16m 21s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 21s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 21s

root: The patch generated 0 new + 26 unchanged - 1 fixed = 26 total (was 27)

+1 :green_heart:
mvnsite
2m 40s

the patch passed

+1 :green_heart:
javadoc
1m 53s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 44s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
4m 15s

the patch passed

+1 :green_heart:
shadedclient
34m 49s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 53s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  0s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

256m 49s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7094/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7094

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 5930c9f57926 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.4.1 / eeafe59abe12a8653da207b98dd6418d28f7fd5d

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7094/1/testReport/

Max. process+thread count
1953 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7094/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


fix: CVE-2020-8908 - change guava version to 31.1-jre

This change fixes CVE-2020-8908 by changing Guava version to 31.1-jre.

Closed - guava is updated to the latest version by dependabot.


Fix race condition in FileAccessRepository

Fix race condition in FileAccessRepository
Today when we allow file accesses after project completion, we avoid disposing the log file stream and relevant to this bug we also avoid nulling out the fileTable and deletedDirectories. outside the lock we enumerate these collections to return a FileAccesses to the plugin. If a file access comes in while we're enumerating, we can get an exception like this:
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at Microsoft.MSBuildCache.FileAccess.FileAccessRepository.FileAccessesState.ProcessFileAccesses(Dictionary`2 fileTable, List`1 deletedDirectories) in D:\a\_work\1\s\src\Common\FileAccess\FileAccessRepository.cs:line 301
   at Microsoft.MSBuildCache.FileAccess.FileAccessRepository.FileAccessesState.FinishProject() in D:\a\_work\1\s\src\Common\FileAccess\FileAccessRepository.cs:line 275
   at Microsoft.MSBuildCache.MSBuildCachePluginBase`1.<HandleProjectFinishedInnerAsync>d__59.MoveNext() in D:\a\_work\1\s\src\Common\MSBuildCachePluginBase.cs:line 495
   at Microsoft.MSBuildCache.MSBuildCachePluginBase`1.<HandleProjectFinishedAsync>d__58.MoveNext() in D:\a\_work\1\s\src\Common\MSBuildCachePluginBase.cs:line 456

On solution would be to put ProcessFileAccesses inside the lock. However, upon thinking about it, I realized that we don't need to avoid nulling out those collections and we should just do it regardless. There are already null checks on these collections when they're added to, so there is safety from that aspect. For the data loss, we anyway only use those collections when the project is reported as complete, so it does not hurt anything to just avoid adding to the collections (by nulling them out) after the project finishes.
I am leaving the avoidance of the log file stream disposal though so we can still get the logging for these files. Also, I added a log line for the project finishing to have a clear delineation between accesses which end up being considered vs those which are missed in the logs.

Is this sufficient for the .NET memory model?


Update scripts/process_implications.py: use Lean parser, output irreducible implications

Update scripts/process_implications.py: to process the parsed implications JSON, and figure out the irreducible implications

Running this gave me All implications: 310354 while my own output is millions higher, am I misunderstanding the output or is that the size of the entire closure?

lake exe extract_implications | ruby -rjson -e 'JSON.parse($stdin.read)["implications"].each { |s| puts s["lhs"][8,10] + "," + s["rhs"][8,10] }' | sort -u > /tmp/imp.csv
ruby scripts/transitive_closure.rb  /tmp/imp.csv | wc -l

Gives me 4.5 million, but I would totally believe the error is in scripts/transitive_closure.rb

The figure 310354 is after reducing to the 3557 equivalence classes, without the internal implications in each class and with a single implication between every two related classes. That's probably the source of the difference.

Would it be possible to update the README.md to give links to this data and script, possibly with some warning that the data set is provisional and not guaranteed to be 100% accurate yet?  Having a easily accessible initial data set of implications, even if slightly off from the actual truth, may allow a lot of more data analysis / data visualization oriented people to be able to make some interesting contributions and insights.  (One can also think about adding a chapter to the blueprint to give a longer description of the methodology.)

The figure 310354 is after reducing to the 3557 equivalence classes, without the internal implications in each class and with a single implication between every two related classes. That's probably the source of the difference.

Ah, that makes sense. Given that there's two different graphs to be considering here, pre- and post-condensation, it's probably good to be explicit about it. I'd maybe say 'condensed implications' or something else you find more descriptive to make it clear to others who may be confused like me.


Set-SFTPContent passing weird characters

When I am using Set-SFTPContent to add contents to a file it looks fine before it gets to the correct location. This is what I see windows side when I test with Set-Content to a test text file.
Field-1,Field-2,Field-3,Field-4,Field-5,Field-6,Field-7,Field-8,Field-9,Field-10,Field-11,Field-12,Field-13,Field-14,Field-15,Field-16,Field-17,Field-18,Field-19,Field-20,Field-21,Field-22
When I use Set-SFTPContent and check on the UNIX box the information looks like this:
ï»¿Field-1,Field-2,Field-3,Field-4,Field-5,Field-6,Field-7,Field-8,Field-9,Field-10,Field-11,Field-12,Field-13,Field-14,Field-15,Field-16,Field-17,Field-18,Field-19,Field-20,Field-21,Field-22
Not sure why or how these strange characters at the beginning ("ï»¿") are coming from. Any help would be appreciated!

It is adding BOM to the file in the next release there will be a fix for this #162

On Aug 16, 2017, at 9:39 AM, james30522<EMAIL_ADDRESS>wrote:
When I am using Set-SFTPContent to add contents to a file it looks fine before it gets to the correct location. This is what I see windows side when I test with Set-Content to a test text file.
Field-1,Field-2,Field-3,Field-4,Field-5,Field-6,Field-7,Field-8,Field-9,Field-10,Field-11,Field-12,Field-13,Field-14,Field-15,Field-16,Field-17,Field-18,Field-19,Field-20,Field-21,Field-22
When I use Set-SFTPContent and check on the UNIX box the information looks like this:
ï»¿Field-1,Field-2,Field-3,Field-4,Field-5,Field-6,Field-7,Field-8,Field-9,Field-10,Field-11,Field-12,Field-13,Field-14,Field-15,Field-16,Field-17,Field-18,Field-19,Field-20,Field-21,Field-22
Not sure why or how these strange characters at the beginning ("ï»¿") are coming from. Any help would be appreciated!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/166, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf0Hts-iOgi0pp8Es9rNxsT8dJmOs5xks5sYvErgaJpZM4O47Zc.

Thank you for the quick response!
I am not sure that this is causing the issues we are seeing because only one of the Fields are affected. But wanted to rule that out anyways.
Is there any ETA for the next version?

No ETA at the moment

On Aug 16, 2017, at 9:46 AM, james30522<EMAIL_ADDRESS>wrote:
Thank you for the quick response!
I am not sure that this is causing the issues we are seeing because only one of the Fields are affected. But wanted to rule that out anyways.
Is there any ETA for the next version?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/166#issuecomment-322776073, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf0Hq4XQCk_LY7eQ9783EGH3dYjuCaBks5sYvLDgaJpZM4O47Zc.

I was able to resolve the issue I was having by setting the encoding to ASCII. Thank you for the help.

Version 2.0.2 was pushed and should fix this issue. Reopen if it persists


Update readme conform to current version & other small updates

Small fixes in the readme conform to the current version of the boilerplate. Updated jquery to version 3.1.1 and updated some dependencies to their latest versions.

Thank you :+1:


Recommendation for setting environment when webhook called

Hi,
I'm finding this package useful, and plan on submitting a pull request soon which contains among other things an updated set of event templates for the 2015-10-12 API. I do have a question though: when a test is run, it makes a call to the webhook endpoint using setEndpoint, however when this call is made the webhook will be run using the local environment, which means the local (rather than testing) database is used. I can think of a few different ways to trick Laravel into using the test environment for such purposes, however wanted to ask what the Team TNT team is doing to resolve this issue.
Jason

Hi,
we were debating about the best way to enable this but it seems that everything we come up with is kind of an ugly hack, at least in our opinion. Any way of tricking Laravel is a bit "shady" IMO but I would love to hear your ideas, maybe you have a clean approach we didn't think of yet :)

Laravel 5+ test framework supports making post requests as part of the TestCase class.
See https://laravel.com/api/5.2/Illuminate/Foundation/Testing/TestCase.html#method_post

@rikh42 yap, you're right, in this case you could only use the loadEventData method to load the webhook data and simply make a post request

I've also had this issue in Laravel, solved it by adding a method to TestCase class:
protected function triggerStripeWebhook($event)
{
    $tester = (new WebhookTester())->setVersion('2018-05-21');
    $data = $tester->loadEventData($event);

    return $this->call('POST', route('stripe.webhook'), [], [], [], [], $data);
}

Now I can trigger a webhook with $this->triggerStripeWebhook('charge.succeeded'); in my tests.


Can't require() zeroclipboard

According to this entry this should work?
npm install zeroclipboard
node
> require('zeroclipboard')
TypeError: Cannot read property 'currentScript' of undefined
    at _getCurrentScriptUrll/node_modules/zeroclipboard/dist/ZeroClipboard.js:216:18)

Additionally, when I try to require it in the browser with a webpack build, I'm getting Uncaught Error: Cannot find module "zeroclipboard", but I'm not sure if that's related to the same issue or not.

ZeroClipboard is heavily bound to the DOM, so loading it in Node.js doesn't really make any sense (nor is it supported).
Can't speak to WebPack with any experience but we've certainly been around the block with AMD and CommonJS users before (RequireJS, curl.js, node-webmake, Browserify, etc.) but -- AFAIK -- we are still functional in all such environments.

In terms of testing (reactjs) components - we can certainly stub out the component that uses zeroclipboard, but as it's in the dependency tree already I suspect it would break any component tests.
There's probably some boilerplate with jsdom that will make this work (it at least needs document, window.location, and at least one script tag in jsdom), but it would be nice if the script require worked regardless. Maybe even just bail if there's no browser / window?

I see no reason why a non-functional script should just bail (i.e. pretend nothing is wrong)... if anything, I would have it throw an Error immediately if such a state is detectable.  However, if you have JSDOM loaded as well, that state will not be detectable. :cry:

Yeah, bailing probably isn't ideal. There are modules designed to work in the browser, such as jquery, which you can require without error in node land. The issue is that zeroclipboard reads many things from the document / window at load time which doesn't seem ideal to me. That means simply requiring the library has side effects. A solution might be to instantiate all the variables based on window etc when you instantiate a new Zeroclipboard(). If you add syntax to support require and publish it to npm, I don't think it's unreasonable for users to think they can require it in nodejs.
If it errors at require() time in node land, it makes it impractical to unit test that component, because you can't even require the component itself to stub out dependencies. React components are obviously designed to be run in the browser, but they are also very unit testable in node land, unless some dependencies won't load :(

This did indeed break our unit test suite, as a temporary workaround we're doing something like:
NODE_ENV=testing testcommand ...

And in your source:
var ZeroClipboard = process.env.NODE_ENV === "testing" ? function() {} : require("zeroclipboard");

@DelvarWorld: Can you offer up an example of a React component source code that demonstrates how they are making them work well for testing in both the browser and in Node? That would be very helpful. Thanks!

https://facebook.github.io/react/docs/test-utils.html ?

The best friends club:

https://github.com/ccampbell/mousetrap/issues/283
https://github.com/zeroclipboard/zeroclipboard/issues/568
https://github.com/thlorenz/brace/issues/30


SimpleSchema custom validation and current document ID

Hi, this is part-question, part-feature request. I asked this question on the forum but since I didn't get a reply let my ask it once again here...
In short - when running custom validation on the server, is there a way to access the current document ID? (the document being validated currently, when the operation is an update)
If not - then would you be able to advise an alternative to my approach?
And/Or please consider this as a feature suggestion.
Here's the full question from the forum, along with the use case motivation.
I implemented a custom validation function and in this function I need to access the current document ID. Is that possible?
If it's an insert (new document) - the document ID should be null, but if it's an update - I'd like to use the document ID in order to run the custom validation.
According to the doc, the document ID doesn't seem to be provided (https://github.com/aldeed/meteor-simple-schema#custom-validation)
Here's my scenario, maybe there's another way around this.
I have two types of collections: Teams and Companies. A team belongs to a single company (team.companyId) and a company may have many teams. The app is here btw in case you want to get the hang of it: http://teams.gormim.com
Each team has a vanityUrl for example "google/gmail" or "google/search"
Each company also has a vanity URL, for example "google" or "microsoft".
When validating the team's vanityUrl field I'd like to make sure that the URL is unique, that it isn't used yet.
This is fine with new teams (or new companies) as the object gets inserted, I run a query over the DB and check if any other team already is using this vanity URL.
But for updates - if I run this query, the result includes this current team's vanity URL, so in cases where the vanity URL isn't changed (e.g. other fields of the document are changed) I get an error - since vanity URL already exists, it is this document's vanity URL.
If I had access to the current document ID (the current document that's being validated) I'd be able to exclude this result, but not having access to the current document ID - I'm at loss...
On top of that, there are some business rules regarding the relationships b/w a team and a company, so for example it's OK to have a team's vanity URL "similar" to a company's URL in some sense (the precise logic isn't important).
Here again - if I had the team ID when validating team.vanityUrl that would be great b/c from the teamId I can easily query the DB (server side of course) and find out the company.
So to sum up - I wish that using SimpleSchema's custom validation, I'd have access to the current document ID, at least when we're talking about updates.
Is this possible?
thanks!

this.docId should be set if collection2 package is handling the update.

Hmm, so simple, how did I miss that?
But I noticed that while on the server this.docId is valid, on the client it's undefined. Is this by design?

For an insert? Client validation probably runs before the mongo pkg code that generates the _id. But it should be set for an update.

No, it's actually for an update, not an insert and this.docId is undefined on the client.
So you're saying it is supposed to be set on the client as well?
OK, I'll try to get a repro sometime to send you (or find our where I'm wrong :wink:)

Yes https://github.com/aldeed/meteor-collection2#custom

OK, repro isn't easy, on a barebones meteor project I was not yet able to repro this, but on my main repo (large and a private one) I'm able to consistently reproduce this.
The behavior is that on the client side this.docId is undefined. On the server this.docId is fine (well defined).
However, while working to repro this on a minimal barebones project, I went into the main project and upgraded the world. (meteor to <IP_ADDRESS> etc) and while doing to I also upgraded simple-schema related packages, which surprisingly fixed this bug for me!
Here's the list of upgrades related to simple schema (one of them probably fixes my problem):
<EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>
So as far as I concern the problem is solved ;)


ENQU: clear fields #1110

Removed fields that are taken from field definition of the table/view

#1110


Refactor the Network component to hold different networks

Depends on #651.
Refactor code to create a Networks wrapper type that groups Network components by chain id.
This allows selecting the appropriate Network value based on the protocolNetwork property for every context.
E.g., Action Workers must resolve their Action's protocol network before executing them.

Resolved by #668


How to use the cell formatter to return a Svelte component?

Basically something like https://gridjs.io/docs/examples/react-cells
Thanks! :)

"Grid.js uses Preact to render the elements and that means that you can take advantage of Preact's Virtual DOM and render complex cells."
And Svelte approach is using the compiler and uses real DOM, instead of using virtual DOM. So I think, you can't use Svelte component to handle this.
But you can use html, check out my repl.

Hey, thanks for the reply!
html

Works fine if you just want to return the cell itself, but not a svelte component. There must be a way to create a Svelte component on fly, pass the props and then return the built html. I'll try to play around with it, thanks so far! 👍

+1.
Would be great to be able to use a Svelte component so that changes to the site theme are automatically reflected in gridjs instances.

I would like to use svelte components with gridjs so that I can use bootstrap icons with my data.  It would be very useful to use something like html from gridjs to do it.

Hey there! On v2 I already added plugins to wrapping the svelte component, so now we can to svelte component to render the cell. Check out the example.
Note: (currently!) when we use plugin we not receive the data, I don't why but I think it's might bug on Grid.js


[BUG] HA 3 replica podAntiAffinity issue for longhorn-ui, longhorn-recovery-backend, longhorn-conversion-webhook, longhorn-admission-webhook

Using helm to deploy HA longhorn with 3 replicas - one each on 3 dedicated storage nodes.  longhorn-ui, longhorn-recovery-backend, longhorn-conversion-webhook & longhorn-admission-webhook all deploy all 3 replicas on the same node (12 pods on one storage node!) making that node unavailable for the remaining longhorn pods that are still deploying.  I end up with longhorn-csi-plugin unable to deploy on that node (storage-01).  csi-* pods all deploy with podAntiAffinity the way they should.
longhorn-system   longhorn-admission-webhook-798f4c6b8b-2c625             1/1     Running     storage-01
longhorn-system   longhorn-admission-webhook-798f4c6b8b-hlxmx             1/1     Running     storage-01
longhorn-system   longhorn-admission-webhook-798f4c6b8b-k56kd             1/1     Running     storage-01
longhorn-system   longhorn-conversion-webhook-54946896dd-7svm6            1/1     Running     storage-01
longhorn-system   longhorn-conversion-webhook-54946896dd-bnh4p            1/1     Running     storage-01
longhorn-system   longhorn-conversion-webhook-54946896dd-wgrml            1/1     Running     storage-01
longhorn-system   longhorn-csi-plugin-dkdhj                               3/3     Running     storage-02
longhorn-system   longhorn-csi-plugin-ncgfp                               0/3     Pending     <none>
longhorn-system   longhorn-csi-plugin-rpjnf                               3/3     Running     storage-03
longhorn-system   longhorn-driver-deployer-67f94f947f-h8nwk               1/1     Running     storage-01
longhorn-system   longhorn-manager-gxfv7                                  1/1     Running     storage-03
longhorn-system   longhorn-manager-gznj6                                  1/1     Running     storage-01
longhorn-system   longhorn-manager-sm248                                  1/1     Running     storage-02
longhorn-system   longhorn-recovery-backend-5ccf9cc847-9nt2p              1/1     Running     storage-01
longhorn-system   longhorn-recovery-backend-5ccf9cc847-9vj7k              1/1     Running     storage-01
longhorn-system   longhorn-recovery-backend-5ccf9cc847-hblfh              1/1     Running     storage-01
longhorn-system   longhorn-ui-7dcddc7c45-5cz92                            1/1     Running     storage-01
longhorn-system   longhorn-ui-7dcddc7c45-rpfxw                            1/1     Running     storage-01
longhorn-system   longhorn-ui-7dcddc7c45-tdqzm                            1/1     Running     storage-01

A clear and concise description of what the bug is.
Trying to figure out why podAntiAffinity is not working on the 4 pods described.  Taking longhorn-ui as an example - when looking at the helm chart I can see that deployment-ui.yaml has podAntiAffinity for the pod.  It just isn't working for my deployment and I'm not sure why.
spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - longhorn-ui
              topologyKey: kubernetes.io/hostname

To Reproduce
Steps to reproduce the behavior:

Deploy with 3 dedicated storage nodes, setting nodeSelector and tolerations for the dedicated nodes in question.
Wait for all pods to deploy in HA configuration.
Notice 12 pods (3 of each) all deployed on the same node.
See pending status for longhorn-csi-plugin on overallocated node.

Expected behavior
I expect longhorn pods to follow conventions for podAntiAffinity on all pods deployed through helm which for a 3 replicaset deployment means one pod per node.
Configs
values:
      defaultSettings:
        createDefaultDiskLabeledNodes: true
        upgradeChecker: false
        replicaAutoBalance: best-effort
        systemManagedComponentsNodeSelector: "node-role.kubernetes.io/storage:storage"
        taintToleration: storage=true:NoSchedule
      longhornAdmissionWebhook:
        replicas: 3
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"
      longhornConversionWebhook:
        replicas: 3
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"
      longhornDriver:
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"
      longhornManager:
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"
      longhornRecoveryBackend:
        replicas: 3
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"
      longhornUI:
        replicas: 3
        nodeSelector:
          node-role.kubernetes.io/storage: storage
        tolerations:
          - key: "storage"
            operator: "Equal"
            value: "true"
            effect: "NoSchedule"

Environment

Longhorn version: 1.4.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Ansible Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: RKE2 v1.26.4+rke2r1

Number of management node in the cluster: 3
Number of worker node in the cluster: 3 + 3 dedicated storage nodes

Node config

OS type and version: Ubuntu 22.04
CPU per node: 4-5
Memory per node: 4GB
Disk type(e.g. SSD/NVMe): SSD
Network bandwidth between the nodes: Same motherboard/CPU/RAM running VM's on XCP-ng

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Xen/Baremetal
Number of Longhorn volumes in the cluster: 3?

Additional context
I also have a LimitRange set for all pods in the longhorn-system namespace for testing pod limits.  If I remove the LimitRange the longhorn-csi-plugin will deploy just fine, but thats not the issue here.
apiVersion: v1
kind: LimitRange
metadata:
  name: longhorn-resource-limit-range
  namespace: longhorn-system
spec:
  limits:
  - defaultRequest:
      memory: 128Mi
      cpu: 150m
    default:
      memory: 200Mi
    type: Container

Typically, I like to use requiredDuringSchedulingIgnoredDuringExecution: when using podAntiAffinity.  I noticed the longhorn-ui uses preferredDuringSchedulingIgnoredDuringExecution:.

Hi @Jamison1
Can you provide the support bundle?  We can help to check the the cluster situation.
You can also try to delete one of the webhook pod to see if it is redistributed to other nodes
and observe the pod event to see how kubernetes schedule it.
Thanks

I deleted one pod longhorn-ui and it redistributed to a different storage node.  I deleted it again and it redistributed to the 3rd storage node.  So there is no issue with the pod installing on all 3 storage nodes.
I also reduced replicas to 2 for this test.
The issue is only at install when the first storage node is available.  This is what it looks like at install with the first batch of longhorn related pods:
longhorn-system   longhorn-admission-webhook-f97d66bd-d6sm5              0/1     Pending     0          4s
longhorn-system   longhorn-admission-webhook-f97d66bd-p9xl4              0/1     Pending     0          4s
longhorn-system   longhorn-conversion-webhook-6d9b6f4467-5f8g8           0/1     Pending     0          4s
longhorn-system   longhorn-conversion-webhook-6d9b6f4467-phv72           0/1     Pending     0          4s
longhorn-system   longhorn-driver-deployer-844945f4f6-xff7c              0/1     Pending     0          4s
longhorn-system   longhorn-recovery-backend-5f494cb46d-bgvx9             0/1     Pending     0          4s
longhorn-system   longhorn-recovery-backend-5f494cb46d-jhp7h             0/1     Pending     0          4s
longhorn-system   longhorn-ui-6b78cfc64d-c2fsw                           0/1     Pending     0          4s
longhorn-system   longhorn-ui-6b78cfc64d-hn8b9                           0/1     Pending     0          4s

Every one of those gets thrown onto the first storage node that is available.  After a few seconds the rest of the longhorn pods start deploying and they respect the podAntiAffinity across all 3 nodes.
In looking into this further I examined deployment-ui.yaml within the helm chart and found that podAntiAffinity is set as soft antiAffinity preferredDuringSchedulingIgnoredDuringExecution: instead of hard antiAffinity requiredDuringSchedulingIgnoredDuringExecution:.  This may be what is affecting it as it seems to only see one storage node initially.
I've had this issue with other helm installs with HA 3 replicas over the past several weeks and after setting podAntiAffinity to required, the issue went away in every case.
Perhaps there is a way to enable hard podAntiAffinity through helm values.yaml file with requiredDuringSchedulingIgnoredDuringExecution:.
Interesting that pod anti-affinity works for 3 replicas of csi-attacher, csi-provisioner, csi-resizer, csi-snapshotter deployments which are deployed later in the helm install.

I assume is a timing issue?
Maybe there is only one node available when doing helm install so the first batch of pods were scheduled to the first node.
CSI related pods were deployed by longhorn-driver-deployer not in the helm chart, so when the longhorn-driver-deployer was ready to deploy CSI related pods, other nodes were also ready as well.

Hi @innobead , do you know why we use preferredDuringSchedulingIgnoredDuringExecution  instead of requiredDuringSchedulingIgnoredDuringExecution?


Moar Azure Devops

GitHub Actions has been sufficient, and also supports Windows builds for when we need them


int() cannot deal with scientific notation

Don't use parseInt, if you supply a number in scientific notation (e.g. a very large or very small number), it will convert to string first and then parseInt(2.08e-11) = 2.
Consider tweaking String.toNumber in enketo-xpathjs so it can convert "2.08e+3" to 2808, but check native XPath evaluator first.

2 is not feasible because the browser's built-in XPath evaluator is compliant with XPath 1.0 (checked Chrome)


ADD CONTACT INFORMATION ON README

📋 Description
Added contact information to the README file to improve communication and feedback channels for users and contributors.
🔨 Changes Made

Updated the Table of Contents to include a "Contact Information" section.
Added a "Contact Information" section to the README with email details.
Ensured the contact information is clear and accessible for users.

✅ Checklist
Before submitting the PR, please make sure you have completed the following:

[x] I have followed all the guidelines mentioned in CONTRIBUTING.md.
[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation (if applicable).
[x] I have tested it locally and it works fine.
[x] Any dependent changes have been merged and published in downstream modules.

🏷 Types of Changes
What type of changes does your code introduce? (Check all that apply)

[ ] Bug fix (non-breaking change which fixes an issue) 🐛

[x] New feature (non-breaking change which adds functionality) ✨

[ ] UI enhancement (non-breaking change which enhances UI) 🎨

[x] Documentation update 📚

Fixes #227

📸 Screenshots (if applicable)

Thank you for contributing to Research Nexus! We look forward to reviewing your PR and getting it merged!

@Harshdev098 , the task has been completed.

Done @Harshdev098 , you can review it!


possible to use with stability-ai/sdxl?

Would it be possible to replace the training model with:
https://replicate.com/stability-ai/sdxl?

saw #32 but not exactly the same as loading to sdxl within replicate

Hi @lopezi, I did a try with sdxl and the results so far are not as good as sd1.5 (with dreambooth or Lora)


Updated version to 3.4.0

Codecov Report

Merging #574 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #574   +/-   ##
=======================================
  Coverage   59.24%   59.24%
=======================================
  Files          88       88
  Lines        4002     4002
=======================================
  Hits         2371     2371
  Misses       1631     1631

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a75d15c...6febbf3. Read the comment docs.


Add support of Min/Max/Avg latency in flow metric

Added minimum-latency, maximum-latency and average-latency in following model:
Support for these attributes is already available in OTG models.
module: open-traffic-generator-flow
  +--rw flows
     +--ro flow* [name]
        +--ro name     -> ../state/name
        +--ro state
           +--ro name?              string
           +--ro transmit?          boolean
           +--ro loss-pct?          otg-types:ieeefloat32
           +--ro out-frame-rate?    otg-types:ieeefloat32
           +--ro in-frame-rate?     otg-types:ieeefloat32
           +--ro out-rate?          otg-types:ieeefloat32
           +--ro in-rate?           otg-types:ieeefloat32
           +--ro minimum-latency?   otg-types:timeticks64
           +--ro maximum-latency?   otg-types:timeticks64
           +--ro average-latency?   otg-types:timeticks64
           +--ro counters
              +--ro in-octets?    otg-types:counter64
              +--ro in-pkts?      otg-types:counter64
              +--ro out-octets?   otg-types:counter64
              +--ro out-pkts?     otg-types:counter64

Looks good to me. Just a general query : What happens today if the underlying implementation does not support it ... it will simply not return a value for these fields ?

These 3 fields are supported today in OTG Model and Ixia-c controller. Implementation in Ixia-C implementation is to simply return USstream stats for the corresponding fields, which are always filled up with some value. So, assuming, if the underlying implementation does not support these params, then also it will be always populated - might be with some initialization value of 0. Need to validate ixia-c behavior / if any known issue from Ixia-C.


[Edison's Ball] Updated for BangleJS 2

Updated Edison's Ball app to work on Bangle JS 2

Thanks!


feat(docker): create configs for dockerized setup

This intends to create a dockerized setup that we can build and push to our private registry. This will follow up with implementation to use Azure App Service to deploy containers continuously.
Few things to do when this is being merged:

[ ] Add detailed documentation about doppler in the staff-wiki.
[ ] Create and test configs for various languages we support.
[ ] Add test setup for local builds.
[ ] Investigate better caching while building.

Note: This will also make it so that the client config repo is no longer required.

Additional notes on testing:
You will be needing access to doppler and have your workspace set up.
 Also merging this should not be affecting our regular workflow in any case.

Not blocking, but I wondered about something. We're currently using pm2 as a process manager for the api and the client, but this PR keeps using pm2 for the api, but not for the client.  Is that intentional or should we be relying on Azure for that?
I'm assuming Azure will restart the client container if it exits, but I've not looked into it.

.... Is that intentional or should we be relying on Azure for that?
I'm assuming Azure will restart the client container if it exits, ...

It is intentional at least at this point to avoid having to touch the API code. This way we are swapping a VM in favor of a container and that's about it - we have a good understanding of the PM2 behavior over the years and hence the path with the least friction.
For the client, we have vetted that serve works natively (from the news deployments) and there is nothing to crash, in our current setup we have had to warp it up in a shell file.

Thanks for the context, Mrugesh. That all makes sense - particularly deferring to another PR.


Always index only one version's source files

Description
If the default version for an entry is not set, we were indexing all the source files from all versions. With this fix, we only index the source files from one version.
There were some workflows that were failing to index because all their versions' source files are greater than 10MB, and the ES we've been using in AWS has a 10MB request limit. We are temporarily upgrading to an (expensive) AWS ES instance type that allows 100MB requests, but once this merged, we can scale down ES again.
The version to index when there is no default uses the same logic as the UI, by taking the most recently created valid version (the highest id). The only difference is it does default to the latest version if there is no valid version, which is theoretically possible.
On doing the processing of workflow versions in Java rather than with HQL, which would be a ton more efficient:

This is a hotfix, I want to limit the scope of the change
We were already processing the versions in Java
This code can go months without being executed -- we typically only need to reindex if we change the schema (or this bug), so it's arguable whether it's worth the effort in general.

Also, I think we should just always set the default version -- I'm creating a separate issue for this.
Issue
#5154
Please make sure that you've checked the following before submitting your pull request. Thanks!

[ ] Check that you pass the basic style checks and unit tests by running mvn clean install
[ ] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[ ] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[ ] If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
[ ] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[ ] Do not serve user-uploaded binary images through the Dockstore API
[ ] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[ ] Do not create cookies, although this may change in the future
[ ] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

A) why not break up requests to get under 10MB? B) if theoretically, a workflow had more than 10MB of descriptors in one workflow version, wouldn't we still have the problem?

We do try to break up the requests to a max of 2.5MB. But

The code only creates a new request after the size of the current request exceeds that limit, but doesn't reduce the size of the request, it just creates a new request after that. That does seem lame to me; but that's how it looked when I debugged it.
With the current code, we actually do have some workflows exceeding 10MB, with source files from all versions. Those can't be broken up in the current model. We probably could/should break up the requests further so that would work, but that would require a deeper understanding of ES than I currently have. There are currently no entries where a single version causes more than 10MB to be indexed, so I feel this is pretty safe for the hotfix.

To clarify, the pseudo code with 4 entries with requests of 1MB each is:

Add 1MB from first entry to payload. Total payload of the request is not over 2.5 MB, go to next entry.
Add 1MB from second entry to payload. Total payload of the request is not over 2.5 MB, go to next entry.
Add 1MB from third entry to payload. Total payload of the request IS over 2.5, execute the now 3MB request, even though it's more that 2.5MB,  and prepare a new one.
Add 1MB from the 4th entry to a new payload.

The code only creates a new request after the size of the current request exceeds that limit, but doesn't reduce the size of the request, it just creates a new request after that. That does seem lame to me; but that's how it looked when I debugged it.

Hmm, seems like a follow-up issue then.

2. There are currently no entries where a single version causes more than 10MB to be indexed, so I feel this is pretty safe for the hotfix.

Presumably it could though, we're just lucky and/or no one has been quite that crazy yet. It does seem fair to look for a better more general solution in 1.14 though and possibly revert this when we get a better solution


KAFKA-10180: Fix security_config caching.

After 8b22b8159673bfe22d8ac5dcd4e4312d4f2c863c security_config properties are modified during setup_node: tls_version patched according to the node jdk version. But python @property decorator doesn't store property values and fresh instance of SecurityConfig created on each kafka.security_config call
This patch caches SecurityConfig instances so correct tls_version will be used during the test.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Results of the system tests: Down to 13 failures and all benchmark related failures are gone. http://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/2020-06-26--001.1593223563--nizhikov--KAFKA-10180--ba1dee62e/report.html

@rhauch : This PR may fix a bunch of system tests in 2.6 too. Since 2.6.0 is code freeze now, I will let you decide if you want to merge this PR to 2.6.

We need this in 2.6 to ensure we have passing system tests, so I cherry-picked it to that branch.


Support for telemetry

Hi and thanks for an awesome work,
Would it be possible to add telemetry support for instance with BME860 sensor?
Thanks.

Hi Peter,Theoretically it should be possible and easier to add support for the BME680 environmental sensor considering that Adafruit has an ESP32 library for it.https://www.adafruit.com/product/3660https://randomnerdtutorials.com/esp32-bme680-sensor-arduino/The only challenge would be that it will be lower in the list of things to do, because Ricardo the developer would need the sensor for testing and time to do it, because I imagine he has great  and very interesting new enhancements ongoing that currently have higher priorities.Of course feel free to fork the project and add it yourself, and then you can create a pull request to allow merging it back into the source project. Basically new contributors are highly welcome :) .Thanks and 73 from Chile to the Slovac Republic,Tihomir CA3TSKOn May 29, 2023 07:05, Peter Javorsky @.***> wrote:
Hi and thanks for an awesome work,
Would it be possible to add telemetry support for instance with BME860 sensor?
Thanks.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

Sorry, my fault.
Of course I meant BME680, like this one:

But there are many types of course.
I would might be able to make a PR, but I would need help regarding sending the telemetry.
Is there any code that you could prepare if I would fork the repo?
I have several of these sensors in my lab, so it wouldn't be a problem to integrate & test it.

no problem....  let me know what info and how you want to show it and I could to the "sending telemetry" part.
maybe send me a mail directly

Awesome!
Do you have any ongoing/unfinished work on the project that you would like to merge first?
Otherwise, I can fork the repo and start working on the BME680 sensor support - i.e. reading the data first.
Afterward, you could help me with sending the telemetry data from the sensor on my fork, and after that, we can merge it back into your repo.
Not related to this issue, but a Web UI would be also an awesome feature to have.
For example, my Czech fellow "brother" @ok5tvr has made one on his Lora RX iGate, unfortunately, the HTML and all the static web resources are hardcoded into the code and not contained as files on the SPIFFS, but that doesn't matter at the moment.
Here is how it looks:

I can create a separate Github issue for the Web interface if you like this idea, and I could help with that as well... but not promising myself, as I'm extremely busy these times.

as for forking, GO AHEAD!!! there are lots of ideas to implement, and its nice to have some help! ;)
as for the web UI, I really want this, could you put me in contact with your friend as I'm wanting to have a Web configuration for iGate and Tracker also

You can find his e-mail address on his QRZ page after logging in.
You can write him, I don't know him personally, I don't even know if he speaks english... just some friends from our radio club noticed the repo and have tried his iGate.
Hopefully next weeks I'll manage to make some time to implement BME680 readings and then we can proceed further.

Hi, I'm Tomas Vlas OK5TVR - this is my Igate project.
Igate - decodes the aprs icon

decodes the position of the broadcasting station, even from the
compressed GPS position.
calculates the distance and course to the station
transmits telemetric data to aprs (number of packets, maximum distance to
the transmitter station, and esp32 core temperature.

út 30. 5. 2023 v 12:02 odesílatel Peter Javorsky @.***>
napsal:

You can find his e-mail address on his QRZ page
https://www.qrz.com/db/OK5TVR after logging in.
You can write him, I don't know him personally, I don't even know if he
speaks english... just some friends from our radio club
https://integrac.sk noticed the repo and have tried his iGate.
Hopefully next weeks I'll manage to make some time to implement BME680
readings and then we can proceed further.
—
Reply to this email directly, view it on GitHub
https://github.com/richonguzman/LoRa_APRS_iGate/issues/1#issuecomment-1568154017,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQ6NFZZLX7TOLGBYC5BWSO3XIXATZANCNFSM6AAAAAAYSVAITA
.
You are receiving this because you were mentioned.Message ID:
@.***>

so BME280 and BMP280 support has been added, will look at the web  ui and more in a few days ;)

Awesome @richonguzman sorry for not being helpful at all here, but I was terribly sick.
Will check out the functionality later.
Thanks again!

Sorry, my fault.
Of course I meant BME680, like this one:

But there are many types of course.
I would might be able to make a PR, but I would need help regarding sending the telemetry.
Is there any code that you could prepare if I would fork the repo?
I have several of these sensors in my lab, so it wouldn't be a problem to integrate & test it.

Hi TEKK
hope you are feeling better,
I added a new branch adding-BME680 to test and validate the correct use of this module too. Please test

Hi @richonguzman,
I've just successfully compiled your branch and got the data out of the BME680! Yeeey!
The output needs some adjustments, but it'll go.
Here's what I'm getting:

The things I've changed:

in bme_utils.cpp line 29: status = bme.begin(0x77); ... my sensor was on address 0x77 according to I2C scanner, first I was fighting with wiring as I thought this is the problem, but it wasn't, LOL
same file, line 141 changed to humStr = generateHumString(newHum); as type was undefined there
And of course uncommented BME680 sensor type in header file
After that I was able to get the correct output.
Sorry for not making a PR but I'm happy that I managed to make it work at least...
Thinking about how to fit the kOhms on display (maybe use scrolling or switch the texts once per few seconds...)
Also the kOhms value has to be inserted into telemetry properly, but I'm no keen to APRS protocol, so can't provide any advice on that.
But all in all I think this is a great success! Thank you for making this possible!
73 OM7TEK

nice testing, will do some code improvement and se how the Gas could help out also

What would be even more awesome, if I could have a BME860 on my tracker, measuring all the values on the go, strapped on a backpack.

we'll you should check the latest branch  of the tracker ;) its running good (not main yet because testing a few more things)

I tried to add a BME680 to an existing iGate - double checked all the wires and connections here is my diagram:  https://flic.kr/p/2pFudfu  and when the iGate boots it connects to wifi but then I get a  error on the screen BME/BMP sensor active but no sensor found...  and the iGate locks up:  https://flic.kr/p/2pFwAaM  Not much of a coder and I see that OM7TEK got one running with some modifications.   Currently running 2024-03-18  looking for suggestions.

i will try to make a "searchI2Caddresses" procedure to control the changing addressees in BME modules ;)

I2C address search and autoconfig for I2C Wx modules are available on V1.0.3 Web installer


MINOR: Change “Choose Page” to “Search or choose Page”

Based on discussions with James Ford, it wasn’t clear that you could
search.

OK so what do we do here? Do we require a new entity code for every wording change, and accept that we lose the old translations?

For minor changes, we could update existing translation keys and sync with Transifex on that branch. This would mean we don't pull translations into previous patch releases any more, because we'd risk losing translations, or introducing translations incorrect for the old context.
I think the best approach for minor changes would be to add new keys. This way they can safely be pulled into patch releases as well without corrupting old translations.
I also think that whenever possible we should make translation changes in the earliest minor release branch we have and merge them up.
In this case "Choose page" and "Search or choose page" are two different sentences, so I think separating them is the right thing to do.

So what's the conclusion here? Rename CHOOSE_MODEL to SEARCH_OR_CHOOSE_MODEL in our code and do the same thing in Transifex so we retain existing translation?

Have updated to new translation key. This is the best solution IMO.

@robbieaverill If you can state your approval, I'll merge as we're not going to get a green button on this one with the inter-dependent PRs.


Topic 3: Improve clustering by adding an identifier with the schema instcode:catnumber

For the clustering, a the triple ID (inst:coll:cat) is added as an identifier for records. In ENA, for several records the voucher ID is constructed just with instcode and catnumber (inst:cat), missing the collection code. Adding this pattern to the array of used IDs for a specimen would increase the chances of matching with an ENA record.
Datasets affected: https://www.gbif.org/dataset/d8cd16ba-bb74-4420-821e-083f2bac17c2
Example ENA record with inst:cat as identifiers: https://www.gbif.org/occurrence/3349806846
Related specimen: https://www.gbif.org/occurrence/3347309485
From https://www.insdc.org/documents/feature-table
specimen_voucher="[institution-code:[collection-code:]]specimen_id"
Currently (22 September 21) there are at least 518.274 voucherIDs for sequences in ENA based on such a schema which could benefit from this improvement.

I ran a fix for this and it didn't capture many new relationships.
On exploration, I found that NON_CONFLICTING_DATE and NON_CONFLICTING_COORDINATES require one side to be non-NULL.
I will explore changing that to accommodate missing information on both sides.

For the test set we have in the hackathon (non-observation data) addressing null handling, and this inst:catNum means that
Before:
115,174,986 relationships covering 19,140,426 records
After:
143,123,066 relationships covering 25,233,891 records
For ENA this means:
4,350,320 relationships from 678,574 records
4,749,772 relationships from 1,033,068 records
For Plazi publishers this means:
158,133 relationships from 34,657 records
188,200 relationships from 36,897 records

Fixed in main


Upgraded packages on brew broke MalmoPython.so on Mac

Hi,
I realize this issue seems to have popped up multiple times, but I can't seem to figure out what I should do to remedy it on my end.
Recently I went through and upgraded all my packages through homebrew (probably shouldn't have done this without thinking) and now importing MalmoPython in brew python will cause a Segmentation fault: 11. (System python can't even find the image MalmoPython.so.) I tried rebuilding completely and while the build process seems to be fine, anything that requires MalmoPython will segfault (so all the Python ctest tests fail).
I am still working from 0.30.0 and I at least know my xerces-c version was updated from 3.1.4 to 3.2.1. Is there something I should be doing with the versions of my dependencies to get this to work?
Thanks!

Ended up realizing that I had to build malmo using the system python instead of brew python (for some reason brew python just didn't want to work with malmo). Not sure if this will cause problems going forward but it at least works for now.


[JUnit] Raise error on datadog-ci command not found

What does this PR do?
TL;DR: Working version of #29458.
I fixed the unit test #29458 broke.
This PR raises an error when the datadog-ci command is not found.
Motivation
I stumbled upon a pretty cryptic error while trying to upload the JUnit files to the backend, while working on a runner without datadog-ci installed. I couldn't find quickly what the problem was, so I made it clearer.
Describe how you validated your changes

Possible Drawbacks / Trade-offs
Additional Notes

/merge


Not reproducible after setting random seed

I am running examples/qm9_nn_conv.py
but after setting random seed as follow, the results are still not 100% same.
   def seed_everything(seed=1234):
       random.seed(seed)
       torch.manual_seed(seed)
       torch.cuda.manual_seed_all(seed)
       np.random.seed(seed)
       os.environ['PYTHONHASHSEED'] = str(seed)
       torch.backends.cudnn.deterministic = True
       torch.backends.cudnn.benchmark = False

Hi and thank you for this issue.
Did you notice reproducibility issues only in qm9_nn_conv or everywhere?

I only ran qm9_nn_conv

I evaluated this problem and see perfect reproducibility on CPU when using the seed_everything code from you. However, on GPU, we can not guarantee determinism because we make heavy use of scatter ops to implement Graph Neural Networks which are non-deterministic by nature on GPU.


How i made class diagram

i didnt have to use a plugin to create class diagram, i had to find an alternative since my sequence diagram didnt give me access

i used an in app function on intellij to produce a class diagram.
i was assisted with this video https://www.youtube.com/watch?v=yCkTqNxZkbY&ab_channel=TeacherMarcos


Added handler to SIGTERM to exit with 0 in particle test

Motivated by the need of running the particle example during a certain amount of time and make the test to a controlled shutdown and return 0.
In our test we used the timeout command:

Before these changes, the run failed after the timeout command
Using a fork with these changes, the run succeeded

It can be manually tested by:
timeout --preserve-status 5 ./uniformly_accelerated_particle_demo
echo $?

A blanket trap for signals is a bit of a sledge hammer. @j-rivero do you still need this? There is already a simulation_time command line argument.

Closing. Use the simulation_time or reopen if absolutely necessary.


Transition / Update

Is there an existing method for updating data, and transitioning the SVG polygon to the new position? Thanks!

Would be interested in such a feature as well :+1:


Thing can now have keys other than string

Thing can now take any object for the Key and will close #89.
Fixed some issues with some unicode characters not being considered to be a 'complex character'.
It also fixes some issues from changes in the nightly build. However there seems to be an issue with modify that I have not been able to figure out, @ProphetLamb you might be able to figure out what is going wrong there.
I had to disable a bunch of tests because of the following SurrealDB bug surrealdb/surrealdb#1364

I'll take a look at it, thanks

Might have to just always use JSON objects as keys. Even for primitives, such as integers and strings. The overhead for serialization  is pretty minimal in those cases

@ProphetLamb Got all the tests to go green on my windows machine.
There is something weird with the RestGuidQueryTests in the builds, I'm wondering if they go red on your machine too?

On the other hand #97 just passed which is based off of this branch. 🤔

Maybe they fixed it in the meantime? I'll run the action again

Looks like that did the trick!


CLI app using gocui library.

Hi. I've created an ascii to diagrams convertor CLI application using the gocui library. You can check it out here: https://github.com/esimov/diagram

Added to README via eb73455bec6f3eff128481351bf476915bf95228! :)


Move to Scala 3.2.+

Scala 3.2.+ contains some useful features for Iron including the removal of the "opaque types + inline" implementation restriction.
Since 3.2.+ is not binary compatible, this is considered as a breaking change.

Waiting for 3.2.1 release to use the new Scaladoc. This does not break compatibility with 3.2.0


Creating the team y judgeassessment features

Adding new features:
-Adding the evaluation table for judges.
-Adding a component for judges to view hackathons.
-Creating a page where projects will be loaded for judges to evaluate.
-Creating a new endpoint where users will be loaded, specifying who is a judge.
-Adding a profile for the team to view their information.
-Adding a default image in case the team doesn't upload any image.
Demo of team profile
https://www.loom.com/share/5ae816b84d11498a9c8d4880e84e9724

Everything seems ok for me

Adding new features:

adding the 'judge' table in the database with its foreign key
fetching data to get the projects that have entered the hackathon,
adding the design for displaying the projects.
DEMO
https://www.loom.com/share/e3e6fe0c1b134e448af484070f27c996


chore: log current Quickwit version

Description
Log current Quickwit version on startup
How was this PR tested?
Launched Quickwit with the change

Related issue
https://github.com/quickwit-oss/quickwit/issues/4526

I did the change in a most straightforward way. Just would like to know if it would make sense to create a function in BuildInfo, something like
    pub fn get_version_text() -> String {
        let build_info = Self::get();
        format!(
            "{} ({} {})",
            build_info.cargo_pkg_version, build_info.cargo_pkg_version, build_info.commit_date,
        )
    }

so it can be used in a couple of other places

@etolbakov yes can you add a method on BuildInfo?

Thank you @etolbakov