Split Nearby into a fast query for coordinates + a details query for each pin

Nearby is too slow to use in Paris:

Be in the center of Paris (or use a fake GPS app to pretend)
Open Nearby
Spinning forever, never finishes

I suggest splitting the current big SPARQL requests into:

a first simple request that only returns QID,coordinates pairs.
for each pin, an asynchronous request that loads the place's details: name, description, class, existence, etc.

I did some tests (both with 1km radius):

Request currently performed by the app: Timeout after 60 seconds
Simplified request: Returns all results in 11 seconds

Due to radius stepping, the request that fetches coordinates is often performed several times, making it even more important to fine-tune.
Fetching details afterwards means that some pins will disappear when Exists is enabled, but I don't think it is problematic as it is disabled by default, users who willingly enabled it will understand why the pins disappear.
Latest master.

Hi @nicolas-raoul , this is a great idea, thanks! We will try to start implementing it. :)
One concern that we have is that this would result in a very large number of async calls, in typical use. For instance if there are 40 pins, we will be doing 40 calls. Do you think that would be an issue, or is it routinely done with no problems?
An alternative that I was thinking about is:

First request - we fetch everything that is needed for the map to load, but not everything that we currently do. So QID, coordinates, name, place type and existence (for filters), property type (WLM vs non-WLM)
The second request is only triggered when a pin is selected. Here we will fetch description, Commons link, Wikipedia link, etc

What do you think? I reckon it wouldn't result in time savings as big as your suggestion, so yours would be the ideal if the number of async calls doesn't become an issue.

I don't think that 40 calls is that bad.
A slightly different idea is to make a second request that get the details of all places together.
I suspect that existence checking is the most expensive. And it will get even more expensive in the future.

Out of curiosity, why is existence so expensive?

Here is an example of what the "second query" I mentioned above could look like: https://w.wiki/3ut$
SELECT ?item ?commonsCategory
WHERE {
  SERVICE <https://query.wikidata.org/sparql> {
    values ?item {
wd:Q100234656
wd:Q100234661
# etc other places found by the first simple request
    }
    OPTIONAL {?item wdt:P373 ?commonsCategory}
    # etc other properties, including optional ones
  }
}

About existence: I only suspect it. I will try to perform some measures tomorrow. But the important measure is between the two queries in the original post, and it clearly shows that showing coordinates could be much much faster :-)

So just to summarise what @nicolas-raoul  said, we will be making 2 calls

This will be the initial call which nearby anyways makes, this will include being a part of the radius expander logic, just with a lesser number of properties. Once this is done, we show the nearby with the pins.
Another call will be made for other properties, this will happen in the background, and unlike the first query this will not block the map and once the results are fetched, we update the existing pins with the info of additional properties.

@nicolas-raoul, please verify if I have understood it properly.

@ashishkumar468 Exactly! 🙂

Please let me know if you need any help with the SPARQL syntax.

Hi @nicolas-raoul  @misaochan , I was trying the implementation for the way Nicolas had suggested. Sharing a sample query for my location. Not sure why, specifically for this id - wd:Q65976956, the query fails, otherwise seems to work.  Sharing the exception
`SPARQL-QUERY: queryStr=SELECT
(SAMPLE(?location) as ?location)
?item
(SAMPLE(COALESCE(?itemLabelPreferredLanguage, ?itemLabelAnyLanguage)) as ?label)
(SAMPLE(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?")) as ?description)
(SAMPLE(?classId) as ?class)
(SAMPLE(COALESCE(?classLabelPreferredLanguage, ?classLabelAnyLanguage, "?")) as ?classLabel)
(SAMPLE(COALESCE(?icon0, ?icon1)) as ?icon)
?wikipediaArticle
?commonsArticle
(SAMPLE(?commonsCategory) as ?commonsCategory)
(SAMPLE(?pic) as ?pic)
(SAMPLE(?destroyed) as ?destroyed)
(SAMPLE(?endTime) as ?endTime)
WHERE {
# Around given location...
SERVICE https://query.wikidata.org/sparql {
values ?item {
wd:Q24947183 wd:Q598985 wd:Q6418713 wd:Q63371376 wd:Q63371413 wd:Q63356612 wd:Q63986772 wd:Q26791672 wd:Q63371403 wd:Q65976956
}
}
 # Get the label in the preferred language of the user, or any other language if no label is available in that language.
 OPTIONAL {?item rdfs:label ?itemLabelPreferredLanguage. FILTER (lang(?itemLabelPreferredLanguage) = "en")}
 OPTIONAL {?item rdfs:label ?itemLabelAnyLanguage}

 # Get the description in the preferred language of the user, or any other language if no description is available in that language.
 OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}
 OPTIONAL {?item schema:description ?itemDescriptionAnyLanguage }

 # Get Commons category (P373)
 OPTIONAL { ?item wdt:P373 ?commonsCategory. }

 # Get (P18)
 OPTIONAL { ?item wdt:P18 ?pic. }

 # Get (P576)
 OPTIONAL { ?item wdt:P576 ?destroyed. }

 # Get (P582)
 OPTIONAL { ?item wdt:P582 ?endTime. }

 # Get the class label in the preferred language of the user, or any other language if no label is available in that language.
 OPTIONAL {
   ?item p:P31/ps:P31 ?classId.
   OPTIONAL {?classId rdfs:label ?classLabelPreferredLanguage. FILTER (lang(?classLabelPreferredLanguage) = "en")}
   OPTIONAL {?classId rdfs:label ?classLabelAnyLanguage}

   OPTIONAL {
       ?wikipediaArticle   schema:about ?item ;
                           schema:isPartOf <https://en.wikipedia.org/> .
     }
   OPTIONAL {
       ?wikipediaArticle   schema:about ?item ;
                           schema:isPartOf <https://en.wikipedia.org/> .
       SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
     }

     OPTIONAL {
       ?commonsArticle   schema:about ?item ;
                           schema:isPartOf <https://commons.wikimedia.org/> .
       SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
     }
 }

}
GROUP BY ?item ?wikipediaArticle ?commonsArticle
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:292)
at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:678)
at com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:290)
at com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240)
at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:273)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
at org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter.doFilter(ThrottlingFilter.java:320)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.wikidata.query.rdf.blazegraph.throttling.SystemOverloadFilter.doFilter(SystemOverloadFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:49)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.wikidata.query.rdf.blazegraph.filters.QueryEventSenderFilter.doFilter(QueryEventSenderFilter.java:117)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.wikidata.query.rdf.blazegraph.filters.ClientIPFilter.doFilter(ClientIPFilter.java:43)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.wikidata.query.rdf.blazegraph.filters.RealAgentFilter.doFilter(RealAgentFilter.java:33)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:503)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:889)
at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:695)
at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at com.bigdata.rdf.sail.Bigdata2Sesame2BindingSetIterator.hasNext(Bigdata2Sesame2BindingSetIterator.java:188)
at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
at org.openrdf.query.QueryResults.report(QueryResults.java:155)
at org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:76)
at com.bigdata.rdf.sail.webapp.BigdataRDFContext$TupleQueryTask.doQuery(BigdataRDFContext.java:1722)
at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1579)
at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1544)
at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:757)
... 4 more
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at com.bigdata.rdf.sail.RunningQueryCloseableIterator.checkFuture(RunningQueryCloseableIterator.java:59)
at com.bigdata.rdf.sail.RunningQueryCloseableIterator.close(RunningQueryCloseableIterator.java:73)
at com.bigdata.rdf.sail.RunningQueryCloseableIterator.hasNext(RunningQueryCloseableIterator.java:82)
at com.bigdata.striterator.ChunkedWrappedIterator.hasNext(ChunkedWrappedIterator.java:197)
at com.bigdata.rdf.sail.Bigdata2Sesame2BindingSetIterator.hasNext(Bigdata2Sesame2BindingSetIterator.java:134)
... 11 more
Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at com.bigdata.util.concurrent.Haltable.get(Haltable.java:273)
at com.bigdata.bop.engine.AbstractRunningQuery.get(AbstractRunningQuery.java:1516)
at com.bigdata.bop.engine.AbstractRunningQuery.get(AbstractRunningQuery.java:104)
at com.bigdata.rdf.sail.RunningQueryCloseableIterator.checkFuture(RunningQueryCloseableIterator.java:46)
... 15 more
Caused by: java.lang.Exception: task=ChunkTask{query=4cd24526-5b55-4f6f-8bcc-2412a097e076,bopId=72,partitionId=-1,sinkId=73,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1367)
at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTaskWrapper.run(ChunkedRunningQuery.java:926)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.bigdata.concurrent.FutureTaskMon.run(FutureTaskMon.java:63)
at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkFutureTask.run(ChunkedRunningQuery.java:821)
... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1347)
... 8 more
Caused by: java.lang.RuntimeException: Headless value factory should not be asked for its namespace
at com.bigdata.rdf.model.BigdataValueFactoryImpl.getNamespace(BigdataValueFactoryImpl.java:85)
at com.bigdata.rdf.internal.encoder.IVSolutionSetEncoder.encodeSolution(IVSolutionSetEncoder.java:386)
at com.bigdata.rdf.internal.encoder.IVSolutionSetEncoder.encodeSolution(IVSolutionSetEncoder.java:292)
at com.bigdata.rdf.internal.encoder.IVSolutionSetEncoder.encodeSolution(IVSolutionSetEncoder.java:285)
at com.bigdata.rdf.internal.encoder.SolutionSetStreamEncoder.encode(SolutionSetStreamEncoder.java:134)
at com.bigdata.bop.solutions.SolutionSetStream.put(SolutionSetStream.java:277)
at com.bigdata.bop.engine.LocalNativeChunkMessage.(LocalNativeChunkMessage.java:213)
at com.bigdata.bop.engine.LocalNativeChunkMessage.(LocalNativeChunkMessage.java:147)
at com.bigdata.bop.engine.StandaloneChunkHandler.handleChunk(StandaloneChunkHandler.java:92)
at com.bigdata.bop.engine.ChunkedRunningQuery$HandleChunkBuffer.outputChunk(ChunkedRunningQuery.java:1699)
at com.bigdata.bop.engine.ChunkedRunningQuery$HandleChunkBuffer.outputBufferedChunk(ChunkedRunningQuery.java:1716)
at com.bigdata.bop.engine.ChunkedRunningQuery$HandleChunkBuffer.flush(ChunkedRunningQuery.java:1739)
at com.bigdata.bop.solutions.PipelinedAggregationOp$ChunkTask.call(PipelinedAggregationOp.java:846)
at com.bigdata.bop.solutions.PipelinedAggregationOp$ChunkTask.call(PipelinedAggregationOp.java:519)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1346)
... 8 more`
This can also be seen by running the query link I have shared. Can anything be done about this?

@ashishkumar0207
Maybe because this item has no description in any language?
I will try to fix the request, but for now could you please just comment out the line (SAMPLE(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?")) as ?description)? That seems to make the request succeed.

@ashishkumar0207 This SPARQL query works in all situations, and returns all information, please use it, thanks 😊 :
SELECT
  ?item
  (SAMPLE(?label) AS ?label)
  (SAMPLE(?description) AS ?description)
  (SAMPLE(?class) AS ?class)
  (SAMPLE(?classLabel) AS ?classLabel)
  (SAMPLE(?icon) AS ?icon)
  (SAMPLE(?wikipediaArticle) AS ?wikipediaArticle)
  (SAMPLE(?commonsArticle) AS ?commonsArticle)
  (SAMPLE(?commonsCategory) AS ?commonsCategory)
  (SAMPLE(?pic) AS ?pic)
  (SAMPLE(?destroyed) AS ?destroyed)
  (SAMPLE(?endTime) AS ?endTime)
WHERE {
  SERVICE <https://query.wikidata.org/sparql> {
    values ?item {
      wd:Q24947183 wd:Q598985 wd:Q6418713 wd:Q63371376 wd:Q63371413 wd:Q63356612 wd:Q63986772 wd:Q26791672 wd:Q63371403 wd:Q65976956
    }
  }

  # Get the label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {?item rdfs:label ?itemLabelPreferredLanguage. FILTER (lang(?itemLabelPreferredLanguage) = "en")}
  OPTIONAL {?item rdfs:label ?itemLabelAnyLanguage}
  BIND(COALESCE(?itemLabelPreferredLanguage, ?itemLabelAnyLanguage, "?") as ?label)

  # Get the description in the preferred language of the user, or any other language if no description is available in that language.
  OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}
  OPTIONAL {?item schema:description ?itemDescriptionAnyLanguage}
  BIND(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?") as ?description)

  # Get Commons category (P373)
  OPTIONAL { ?item wdt:P373 ?commonsCategory. }

  # Get picture (P18)
  OPTIONAL { ?item wdt:P18 ?pic. }

  # Get (P576)
  OPTIONAL { ?item wdt:P576 ?destroyed. }

  # Get (P582)
  OPTIONAL { ?item wdt:P582 ?endTime. }

  # Get the class label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {
       ?item p:P31/ps:P31 ?classId.
       OPTIONAL {?classId rdfs:label ?classLabelPreferredLanguage. FILTER (lang(?classLabelPreferredLanguage) = "en")}
       OPTIONAL {?classId rdfs:label ?classLabelAnyLanguage}
       BIND(COALESCE(?classLabelPreferredLanguage, ?classLabelAnyLanguage, "?") as ?classLabel)
  }

  OPTIONAL {
    ?wikipediaArticle schema:about ?item.
    ?wikipediaArticle schema:isPartOf <https://en.wikipedia.org/>.
  }

  OPTIONAL {
    ?commonsArticle schema:about ?item.
    ?commonsArticle schema:isPartOf <https://commons.wikimedia.org/>.
  }
}
GROUP BY ?item

Thanks for helping with the query @nicolas-raoul ! Pinging @ashishkumar468 instead, I don't think the other username works. ;)

Hi @nicolas-raoul , I just talked with Ashish. There are a few things that we need help clearing up, as we aren't very familiar with SPARQL.

I just realized we are actually running two separate queries, one for Nearby (https://github.com/commons-app/apps-android-commons/blob/master/app/src/main/resources/queries/nearby_query.rq) and one for Monuments (https://github.com/commons-app/apps-android-commons/blob/master/app/src/main/resources/queries/monuments_query.rq). Am I correct in thinking that if we ONLY use nearby_query.rq, we should be able to get all the pins we need including the Monument pins, and then we can filter them client-side?

Could you post the SPARQL for the first query (QID/coords) as well please?

Thanks a lot!

My new suggestion:
Always running this as the first query:
SELECT
  ?location
  ?item
  ?monument
WHERE {
  # Around given location
  SERVICE wikibase:around{
    ?item wdt:P625 ?location.
    bd:serviceParam wikibase:center "Point(4.89 52.37)"^^geo:wktLiteral.
    bd:serviceParam wikibase:radius "1". # Radius in kilometers.
  }

  # Wiki Loves Monuments
  OPTIONAL {?item p:P1435 ?monument}
  OPTIONAL {?item p:P2186 ?monument}
  OPTIONAL {?item p:P1459 ?monument}
  OPTIONAL {?item p:P1460 ?monument}
  OPTIONAL {?item p:P1216 ?monument}
  OPTIONAL {?item p:P709 ?monument}
  OPTIONAL {?item p:P718 ?monument}
  OPTIONAL {?item p:P5694 ?monument}
}
}

It is super-fast and only gives places and their coordinates and their WLM identifier if they have one, which is enough to start showing pins on the map.
... and then the second query from my comment above to get details. This second query is much slower (I now suspect it is due to my implementation of language fallback), so I would advise running it both:

Little by little (for instance 10 items at a time), prioritizing places that are currently visible on the user's screen.
AND when a pin is selected, just for that item.

(if there is no time to implement this, running the second query for all fetched items should work in most places, though)

By the way, I use this map to get an idea of Wikidata places density, it seems that Amsterdam and London are very crowded. If you know precise locations of the most crowded places, please share coordinates, that will be useful for testing :-)

Hi @nicolas-raoul , Thanks for sharing this, one of the other problems we realised that my PR was making separate calls for nearby and monuments while @misaochan  suggests that this can be done in a single call with client side filtering. I am not really comfortable very much with SPARQL queries, would it be possible for you too help with that too (without the split query implementation - for now). What we are planning is to get that in place first and take this one as an improvement.

Yeah, as mentioned above I just realized that we are making two separate queries for Nearby and Monuments, and I'm wondering if that was part of the issue, since we are getting a lot of things (e.g. Commons category) twice. If so, I think it would make sense to try combining the two first, as that would be a much simpler implementation that could get the WLM release out the door in time for the start date. What do you think @nicolas-raoul ?
Pinging @neslihanturan for input as well.

Sure, here is a query that gives everything, coordinates and details and whether it is a WLM monument:
SELECT
  ?item
  (SAMPLE(?label) AS ?label)
  (SAMPLE(?description) AS ?description)
  (SAMPLE(?class) AS ?class) # TODO Is it really used?
  (SAMPLE(?classLabel) AS ?classLabel)
  (SAMPLE(?pic) AS ?pic)
  (SAMPLE(?destroyed) AS ?destroyed)
  (SAMPLE(?endTime) AS ?endTime)
  (SAMPLE(?wikipediaArticle) AS ?wikipediaArticle)
  (SAMPLE(?commonsArticle) AS ?commonsArticle)
  (SAMPLE(?commonsCategory) AS ?commonsCategory)
  (SAMPLE(?monument) AS ?monument)
WHERE {
  # Around given location
  SERVICE wikibase:around {
    ?item wdt:P625 ?location.
    bd:serviceParam wikibase:center "Point(4.89 52.37)"^^geo:wktLiteral. # Longitude latitude
    bd:serviceParam wikibase:radius "0.1". # Radius in kilometers.
  }

  # Get the label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {?item rdfs:label ?itemLabelPreferredLanguage. FILTER (lang(?itemLabelPreferredLanguage) = "en")}
  OPTIONAL {?item rdfs:label ?itemLabelAnyLanguage}
  BIND(COALESCE(?itemLabelPreferredLanguage, ?itemLabelAnyLanguage, "?") as ?label)

  # Get the description in the preferred language of the user, or any other language if no description is available in that language.
  OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}
  OPTIONAL {?item schema:description ?itemDescriptionAnyLanguage}
  BIND(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?") as ?description)

  # Get the class label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {
  ?item p:P31/ps:P31 ?class.
    OPTIONAL {?class rdfs:label ?classLabelPreferredLanguage.  # TODO internationalization
  }

  # Get Commons article
  OPTIONAL {
    ?commonsArticle schema:about ?item.
    ?commonsArticle schema:isPartOf <https://commons.wikimedia.org/>.
  }

  # Wiki Loves Monuments
  OPTIONAL {?item p:P1435 ?monument}
  OPTIONAL {?item p:P2186 ?monument}
  OPTIONAL {?item p:P1459 ?monument}
  OPTIONAL {?item p:P1460 ?monument}
  OPTIONAL {?item p:P1216 ?monument}
  OPTIONAL {?item p:P709 ?monument}
  OPTIONAL {?item p:P718 ?monument}
  OPTIONAL {?item p:P5694 ?monument}
}
GROUP BY ?item

Obviously it times out in crowded places. The example above took 30 seconds even with a tiny radius of 0.1.

Thanks a lot Nicolas!

Hey @nicolas-raoul, Can I work on this issue 🙂

@nicolas-raoul I think this Issue should be Locked for GSOC , as it is mentioned on Phabricator too and other tasks are left for GSOC.

Oh sorry @shashankiitbhu, started working already. Will be sharing pr asap 🙂. Maybe @nicolas-raoul we can have a different issue in GSoC.

@kanahia1 I mean it has not been much time since you were assigned this issue, and honestly, I have also done a a lot of starting work on this for the GSOC proposal that I have submitted since I had to make the timeline for it. So all of us are even till this point, good that you have seen this, it'll save you a lot time working on this.
The thing is this Issue is already mentioned and proposals are already built on that , so it'll be really counter productive to change now.
However, I am waiting for @nicolas-raoul suggestions on what should be done here , so it's best to wait for confirmation, thanks.

Sure, No Worries @shashankiitbhu. Sorry for bothering you 😞

@kanahia1 no problem, and you didn't bother me at all and thanks for understanding what I was trying to explain here.
I am sure there are many other Issues you can work on , in this project and help improve this project.

Sorry for the delay, I was bit out of station. I have already restructured the queries given above so that anyone who works on this issue during GSoC can use these code without any hassle. Thanks to Nicolas Raoul, Neslihanturan, Josephine Lim, Ashish for amazing inputs.
rectangle_query_for_nearby_only_coordinates.rq
SELECT
  ?location
  ?item
WHERE {
  # Around given location
  SERVICE wikibase:box {
    ?item wdt:P625 ?location.
     bd:serviceParam wikibase:cornerWest "Point(84.2 26.3)"^^geo:wktLiteral.
     bd:serviceParam wikibase:cornerEast "Point(84.6 26.6)"^^geo:wktLiteral.
  }
}

rectangle_query_for_nearby_monuments_only_coordinates.rq
SELECT
  ?location
  ?item
  ?monument
WHERE {
  # Around given location
  SERVICE wikibase:box {
    ?item wdt:P625 ?location.
     bd:serviceParam wikibase:cornerWest "Point(80.2 26.3)"^^geo:wktLiteral.
     bd:serviceParam wikibase:cornerEast "Point(84.6 26.6)"^^geo:wktLiteral.
  }

    # Wiki Loves Monuments
  OPTIONAL {?item p:P1435 ?monument}
  OPTIONAL {?item p:P2186 ?monument}
  OPTIONAL {?item p:P1459 ?monument}
  OPTIONAL {?item p:P1460 ?monument}
  OPTIONAL {?item p:P1216 ?monument}
  OPTIONAL {?item p:P709 ?monument}
  OPTIONAL {?item p:P718 ?monument}
  OPTIONAL {?item p:P5694 ?monument}
  OPTIONAL {?item p:P3426 ?monument}
}

query_for_item.rq
SELECT
  ?item
  (SAMPLE(?label) AS ?label)
  (SAMPLE(?description) AS ?description)
  (SAMPLE(?class) AS ?class) # TODO Is it really used?
  (SAMPLE(?classLabel) AS ?classLabel)
  (SAMPLE(?pic) AS ?pic)
  (SAMPLE(?destroyed) AS ?destroyed)
  (SAMPLE(?endTime) AS ?endTime)
  (SAMPLE(?wikipediaArticle) AS ?wikipediaArticle)
  (SAMPLE(?commonsArticle) AS ?commonsArticle)
  (SAMPLE(?commonsCategory) AS ?commonsCategory)
WHERE {
  SERVICE <https://query.wikidata.org/sparql> {
    values ?item {
      wd:Q24947183
    }
  }

  # Get the label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {?item rdfs:label ?itemLabelPreferredLanguage. FILTER (lang(?itemLabelPreferredLanguage) = "en")}
  OPTIONAL {?item rdfs:label ?itemLabelAnyLanguage}
  BIND(COALESCE(?itemLabelPreferredLanguage, ?itemLabelAnyLanguage, "?") as ?label)

  # Get the description in the preferred language of the user, or any other language if no description is available in that language.
  OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}
  OPTIONAL {?item schema:description ?itemDescriptionAnyLanguage}
  BIND(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?") as ?description)

  # Get the class label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {
  ?item p:P31/ps:P31 ?class.
    OPTIONAL {?class rdfs:label ?classLabelPreferredLanguage.  # TODO internationalization
  }

  # Get Commons article
  OPTIONAL {
    ?commonsArticle schema:about ?item.
    ?commonsArticle schema:isPartOf <https://commons.wikimedia.org/>.
  }
}
GROUP BY ?item

// Note for contributors
We can get items' information from first two queries, then we can run third query using item which will give information to populate bottom sheet. I ran the third query it took me 72ms (that is really instant 🤯), I guess User can wait for that much time.


Why when we hide the navigationbar, the tabbar go to the top automatic?

Why when we hide the navigationbar, the tabbar go to the top automatic?

@roshanman  sorry for late response. Yes, look for "hideBar" method which is hiding the navigation bar on swipe

Thanks, I have noticed the method hideBar
func hideBar(notification: NSNotification)  {
        let state = notification.object as! Bool
        self.navigationController?.setNavigationBarHidden(state, animated: true)
}

In this method, we just hide the navigationbar, I don't understand why when we hide the navigationbar, the tabbar go to the top position automatic.
Thank you.

it's done automatically by iOS. Once you hide the navbar, everything inside the main view goes up with animation.

Why my button do not moved when i hide the navbar ?

Hey @roshanman that's a good question. It depends how you build UI. If your constrains linked with NavBar then it will animate up, otherwise it will not move. Check the attachment.
navBar.zip

Thanks for your help. This method worked.
But in your code
lazy var tabBar: TabBar = {
    let tb = TabBar.init(frame: CGRect.init(x: 0, y: 0, width: globalVariables.width, height: 64))
    tb.delegate = self
    return tb
}()

//TabBar
self.view.addSubview(self.tabBar)

you do have not used constrains for tabbar. really strange!

Can we make collection view in TabBar to be scrolled if there are many items in TabBar?

@roshanman  yeah, it was a bad decision. In upcoming update will replace InitWithFrames with constrains


Typescript MatchResult object incomplete

Version: 17.1.0
Noticed the following when it comes to the types; current MatchResults interface only contains the following:
export interface MatchResult {
  succeeded(): boolean;
  failed(): boolean;
  message?: string;
  shortMessage?: string;
  getInterval(): Interval;
}

On inspection, looks like it's missing the following fields:
input:string;
matcher: Matcher;
startExpr:Seq;
getExpectedText(): string;
getRightmostFailurePosition(): number;
getRightmostFailures(): Failure[];

Perhaps this solution will help you figure out the problem click

Thanks, yes! We should update the type definitions. Will try to push something out in the next week or so.


Initializer for Vue sample

PLEASE DO NOT MERGE!!
Description / Motivation

Testing Details

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Hey there!
I'm starting working on a Vue template that will use vite instead of vue-cli
Looking in current template I saw in server/server.vue.config.js this line
config.output.libraryTarget = 'this'; // libraryTarget: 'this' is required for use with Sitecore JavaScriptViewEngine

Can anyone please give some more context? 👀
Vite and other bundlers does not support this as a libraryTarget/format https://www.rollupjs.org/guide/en/#outputformat
It seems this is only a Webpack feature https://webpack.js.org/configuration/output/#type-this
I don't know if the mentioned JavaScriptViewEngine is this one https://github.com/pauldotknopf/JavaScriptViewEngine
But from examples it seems you can have an usual module.exports 🤔


emacs puts all autosaves in the same folder as the file itself

This creates a lot of clutter and problems with version control.
It would be more useful to output everything to the same autosave folder.
It might be handy if location of the autosaves could be changed by a settings file.

fixed with 0d1db543eb7d95b00f23846eac1ecb2d7926b4cb


Lab 03L01, need to refresh browser in order to configure MFA

Module: 3
Lab/Demo: L03L01, Configure Conditional Access policies for WVD (AD DS)
Exercise: 1, Task: 2
Step: 7
Without browser refresh, the MFA policy configuration options are disabled

updated


Protocol v11 XDR files

This PR updates XDR files to be up to date with stellar-core v11. Changes affecting this repository:

Operation type names changes:

manage_offer -> manage_sell_offer,
create_passive_offer -> create_passive_sell_offer,
New manage_buy_offer operation.

Offer IDs are now of int64 type.

For backward compatibility, manage_offer and create_passive_offer type names have not been changed in Horizon. This will be a breaking change in Horizon 0.18.0.
Builders for manage_buy_offer operation has not been added in this PR (new issue: https://github.com/stellar/go/issues/1165).

@poliha it is! It's specified in CAP-0006.

@bartekn this might be useful for you: https://github.com/stellar/stellar-core/pull/2039


Add 'Host' header to fix the watcher

Hi, I have been debugging an issue with https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/50
I've managed to replicate it with this:
require 'kubeclient'

ssl_options = { verify_ssl: OpenSSL::SSL::VERIFY_NONE }
auth_options = {
  bearer_token_file: 'token'
}
client = Kubeclient::Client.new(
  'https://kubernetes/api/', 'v1', auth_options: auth_options, ssl_options: ssl_options
)

watcher = client.watch_pods
watcher.each do |notice|
  puts notice
end

I'm not sure if this is the best way to resolve this, but it works.
Here is the error message I was getting before:
400 Bad Request: missing required Host header
.gem/ruby/2.4.1/gems/kubeclient-2.4.0/lib/kubeclient/watch_stream.rb:27:in `each': Bad Request (Kubeclient::HttpError)

@cben @moolitayer please review/approve.

@mindfulmonk can you squash the PR in a single commit? Thanks.

@mindfulmonk have you also seen this problem with create/get/update/delete, or only watch?
(I'm asking because these use different HTTP libs.)
I'm not able to reproduce the underlying problem this PR seems to address:
Both with http 0.9.8 and 2.2.2, Host header is set automatically even if I give a headers hash without it:
> require 'http'
> puts HTTP.get("https://httpbin.org/headers", headers: {"a" => "b"}).body.to_s
{
  "headers": {
    "A": "b",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "http.rb/2.2.2"
  }
}

Same with RestClient.get.
I'll try to reproduce by actually using kubeclient later.
Can you show the exact URL you pass — is it actually https://kubernetes/api/?  Did you by chance use IP address?  Looks so from https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/50, but then I'm not sure how will this PR fix it?  Does a Host: <IP_ADDRESS> header actually make Kubernetes happy?

I have seen this issue only with watch, for example this works fine:
puts client.get_nodes
I have used the fqdn and tested the API with a curl first to ensure it works.
That being said it fixes the same issue on minishift that I had with using the IP as the fqdn.

According to
https://github.com/httprb/http/blob/256539edd9ad8acf0f8a424114ac9fae6fb44782/lib/http/request.rb#L199-L202
http gem also always set a host header (and if we want to set it ourselves
we maybe should include non-default port as that code does?)
Could you repeat the experiment with logging, before and after your patch
(or at least before, I mostly want to understand why it didn't work)?
Add to Gemfile:
gem "httplog", require: false
and bundle install
Then after requiring kubeclient do:
require 'httplog'
HttpLog.configure { |config| config.log_headers = true }
then do the watch...

Code used:
require 'kubeclient'
require 'rack'
require 'httplog'

HttpLog.configure { |config| config.log_headers = true }

ssl_options = { verify_ssl: OpenSSL::SSL::VERIFY_NONE }
auth_options = {
  bearer_token_file: 'token2'
}
client = Kubeclient::Client.new(
  'https://example-cluster.com/api/', 'v1', auth_options: auth_options, ssl_options: ssl_options
)

watcher = client.watch_pods
watcher.each do |notice|
end

Result without the fix:
D, [2017-05-30T13:38:28.878764 #2114] DEBUG -- : [httplog] Connecting: example-cluster.com:443
D, [2017-05-30T13:38:28.984403 #2114] DEBUG -- : [httplog] Sending: GET http://example-cluster.com:443/api/v1
D, [2017-05-30T13:38:28.984702 #2114] DEBUG -- : [httplog] Header: accept: */*
D, [2017-05-30T13:38:28.984852 #2114] DEBUG -- : [httplog] Header: accept-encoding: gzip, deflate
D, [2017-05-30T13:38:28.984973 #2114] DEBUG -- : [httplog] Header: user-agent: rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111
D, [2017-05-30T13:38:28.985078 #2114] DEBUG -- : [httplog] Header: authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
D, [2017-05-30T13:38:28.985175 #2114] DEBUG -- : [httplog] Header: host: example-cluster.com
D, [2017-05-30T13:38:28.985426 #2114] DEBUG -- : [httplog] Data:
D, [2017-05-30T13:38:29.038511 #2114] DEBUG -- : [httplog] Status: 200
D, [2017-05-30T13:38:29.038979 #2114] DEBUG -- : [httplog] Benchmark: 0.034335 seconds
D, [2017-05-30T13:38:29.039213 #2114] DEBUG -- : [httplog] Header: cache-control: no-store
D, [2017-05-30T13:38:29.039360 #2114] DEBUG -- : [httplog] Header: content-type: application/json
D, [2017-05-30T13:38:29.039489 #2114] DEBUG -- : [httplog] Header: date: Tue, 30 May 2017 12:38:27 GMT
D, [2017-05-30T13:38:29.039613 #2114] DEBUG -- : [httplog] Header: transfer-encoding: chunked
D, [2017-05-30T13:38:29.039861 #2114] DEBUG -- : [httplog] Response:
bunch of json here

D, [2017-05-30T13:38:29.049744 #2114] DEBUG -- : [httplog] Sending: GET https://example-cluster.com/api/v1/watch/pods
D, [2017-05-30T13:38:29.049911 #2114] DEBUG -- : [httplog] Header: Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

D, [2017-05-30T13:38:29.049963 #2114] DEBUG -- : [httplog] Header: Connection: close
D, [2017-05-30T13:38:29.049995 #2114] DEBUG -- : [httplog] Header: Host: example-cluster.com
D, [2017-05-30T13:38:29.050024 #2114] DEBUG -- : [httplog] Header: User-Agent: http.rb/0.9.8
D, [2017-05-30T13:38:29.050070 #2114] DEBUG -- : [httplog] Data:
D, [2017-05-30T13:38:29.050133 #2114] DEBUG -- : [httplog] Connecting: example-cluster.com
D, [2017-05-30T13:38:29.180762 #2114] DEBUG -- : [httplog] Status: 400
D, [2017-05-30T13:38:29.180969 #2114] DEBUG -- : [httplog] Benchmark: 0.130588 seconds
D, [2017-05-30T13:38:29.181151 #2114] DEBUG -- : [httplog] Header: Content-Type: text/plain
D, [2017-05-30T13:38:29.181268 #2114] DEBUG -- : [httplog] Header: Connection: close
D, [2017-05-30T13:38:29.181904 #2114] DEBUG -- : [httplog] Response:
400 Bad Request: missing required Host header
/home/username/.gem/ruby/2.4.1/gems/kubeclient-2.4.0/lib/kubeclient/watch_stream.rb:20:in `each': Bad Request (KubeException)
	from main.rb:16:in `<main>'

Result with the fix
  D, [2017-05-30T13:38:54.801152 #2173] DEBUG -- : [httplog] Connecting: example-cluster.com:443
  D, [2017-05-30T13:38:54.907112 #2173] DEBUG -- : [httplog] Sending: GET http://example-cluster.com:443/api/v1
  D, [2017-05-30T13:38:54.907357 #2173] DEBUG -- : [httplog] Header: accept: */*
  D, [2017-05-30T13:38:54.907464 #2173] DEBUG -- : [httplog] Header: accept-encoding: gzip, deflate
  D, [2017-05-30T13:38:54.907534 #2173] DEBUG -- : [httplog] Header: user-agent: rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111
  D, [2017-05-30T13:38:54.907592 #2173] DEBUG -- : [httplog] Header: host: example-cluster.com
  D, [2017-05-30T13:38:54.907648 #2173] DEBUG -- : [httplog] Header: authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  D, [2017-05-30T13:38:54.907747 #2173] DEBUG -- : [httplog] Data:
  D, [2017-05-30T13:38:54.958801 #2173] DEBUG -- : [httplog] Status: 200
  D, [2017-05-30T13:38:54.958917 #2173] DEBUG -- : [httplog] Benchmark: 0.038313 seconds
  D, [2017-05-30T13:38:54.958970 #2173] DEBUG -- : [httplog] Header: cache-control: no-store
  D, [2017-05-30T13:38:54.958997 #2173] DEBUG -- : [httplog] Header: content-type: application/json
  D, [2017-05-30T13:38:54.959019 #2173] DEBUG -- : [httplog] Header: date: Tue, 30 May 2017 12:38:53 GMT
  D, [2017-05-30T13:38:54.959040 #2173] DEBUG -- : [httplog] Header: transfer-encoding: chunked
  D, [2017-05-30T13:38:54.959103 #2173] DEBUG -- : [httplog] Response:
bunch of json here

  D, [2017-05-30T13:38:54.962331 #2173] DEBUG -- : [httplog] Sending: GET https://example-cluster.com/api/v1/watch/pods
  D, [2017-05-30T13:38:54.962432 #2173] DEBUG -- : [httplog] Header: Host: example-cluster.com
  D, [2017-05-30T13:38:54.962477 #2173] DEBUG -- : [httplog] Header: Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

  D, [2017-05-30T13:38:54.962509 #2173] DEBUG -- : [httplog] Header: Connection: close
  D, [2017-05-30T13:38:54.962527 #2173] DEBUG -- : [httplog] Header: User-Agent: http.rb/0.9.8
  D, [2017-05-30T13:38:54.962554 #2173] DEBUG -- : [httplog] Data:
  D, [2017-05-30T13:38:54.962599 #2173] DEBUG -- : [httplog] Connecting: example-cluster.com
  D, [2017-05-30T13:38:55.103207 #2173] DEBUG -- : [httplog] Status: 200
  D, [2017-05-30T13:38:55.103322 #2173] DEBUG -- : [httplog] Benchmark: 0.140598 seconds
  D, [2017-05-30T13:38:55.103463 #2173] DEBUG -- : [httplog] Header: Cache-Control: no-store
  D, [2017-05-30T13:38:55.103518 #2173] DEBUG -- : [httplog] Header: Content-Type: application/json
  D, [2017-05-30T13:38:55.103549 #2173] DEBUG -- : [httplog] Header: Date: Tue, 30 May 2017 12:38:53 GMT
  D, [2017-05-30T13:38:55.103598 #2173] DEBUG -- : [httplog] Header: Transfer-Encoding: chunked
  D, [2017-05-30T13:38:55.104021 #2173] DEBUG -- : [httplog] Response:

  /home/username/.gem/ruby/2.4.1/gems/http-0.9.8/lib/http/response/body.rb:54:in `stream!': body has already been consumed (HTTP::StateError)
  	from /home/username/.gem/ruby/2.4.1/gems/http-0.9.8/lib/http/response/body.rb:20:in `readpartial'
  	from /home/username/.gem/ruby/2.4.1/gems/http-0.9.8/lib/http/response/body.rb:26:in `each'
  	from /home/username/.gem/ruby/2.4.1/gems/kubeclient-2.4.0/lib/kubeclient/watch_stream.rb:24:in `each'
  	from main.rb:16:in `<main>'

Sorry, missed your reply and forgot about this.
Fascinating.  The log shows exactly same Host: example-cluster.com header sent before and after, yet the result is different.
@ilackarms, you know http and some relevant kubernetes / go http code — any idea what's going on?
Can it be the order of headers matters, requiring Host to be first?
The message seems to come from Go's http lib:
https://github.com/golang/go/blob/88a235042/src/net/http/server.go#L947
which doesn't seem to care about order, req.Header is a Header, alias for map[string][]string.
https://tools.ietf.org/html/rfc7230#section-5.4 says client SHOULD put Host first — isn't a MUST.

I ran into this same exact issue today - when using the fluent-plugin-kubernetes_metadata_filter plugin Fluentd was crashing as a result of trying to setup the watch with the error about the missing host header. I confirmed this patch resolves the issue.

Thanks.  I'm still reluctant to put in a "magic" fix if I can't explain what's the problem and how it helps...  Trying to investigate more... Staring at http code, I really don't see any way it could have missed this.
@JCotton1123 did see this with a numeric IP, or with a domain name?
And what kubernetes version is that?
I'm able to reproduce same headers as mindfulmonk's dump above https://github.com/abonas/kubeclient/pull/249#issuecomment-304867850, but I'm not getting any required Host header errors...
But maybe I should just put something like this in.  Please do nag me if I keep this unanswered :smile:

[ ] I think if we do force the header, it should be host:port.
This patch sends Host: <IP_ADDRESS> where previously it sent Host: <IP_ADDRESS>:8443.  Most servers won't care (as you're already connected to correct port), but in principle this is wrong.

[ ] The patch does make Host header come first (due to ruby Hash keeping order).  I'll try sending https://github.com/httprb/http a patch to put it first per rfc SHOULD...

We're utilizing hostnames. K8s version 1.13. @cben


Can't import the module

I cloned to my local machine.  Just try to run the test.js by typing
node test.js
Got the following errors:
simpleCommands.js:14
return this.getOrDiscoverIP().then(ip => {
^^
SyntaxError: Unexpected token =>
at exports.runInThisContext (vm.js:73:16)
at Module._compile (module.js:443:25)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object. (/Users/andysio/workspace/test/yamaha.js:1:84)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
Now, I have tried to install from npm npm install yamaha-nodejs.  Run the test and it worked fine.  What's the difference with the files comparing to one in npm?

Your nodejs version is probably too old for the latest changes in the file. Any recent version should work fine

Yea. you are right. My node doesn't like the ES6 version.  After updating it, it's fine now.  Thanks.


Error calling JSTimers.callTimers

Hi, i always getting this error, when Metro Bundle reloaded.
but theres no error when build from xcode. can you help me please
thanks

System:
OS: macOS 10.15.3
CPU: (4) x64 Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
Memory: 23.83 MB / 8.00 GB
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 10.16.3 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.11.3 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 13.2, DriverKit 19.0, macOS 10.15, tvOS 13.2, watchOS 6.1
Android SDK:
API Levels: 23, 28, 29
Build Tools: 23.0.1, 28.0.3, 29.0.2
System Images: android-28 | Google Play Intel x86 Atom
IDEs:
Android Studio: 3.5 AI-191.8<IP_ADDRESS>91312
Xcode: 11.3/11C24b - /usr/bin/xcodebuild
npmPackages:
react: 16.9.0 => 16.9.0
react-native: 0.61.5 => 0.61.5
npmGlobalPackages:
react-native-cli: 2.0.1

I can't say this is UI Kitten issue by reviewing this logs. You can see a thread with same issue here


Please invite me to the GitHub Community Organization

Name
Tadashi Amano
Discord Username (if applicable)
shinobi#6939
Additional Context
Thank you

Thank you

It's great having you contribute to this project
If you would like to continue contributing to open source and would like to do it with an awesome inclusive community, you should join our Discord Server and our GitHub Organisation - we help and encourage each other to contribute to open source little and often 🤓 . Any questions let us know.

Invitation sent for the GitHub Organisation. Welcome to the community 🎉#
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/jaidevstudio/people
Like this

Tips for practising:
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps

Please check your email or organisation page 🙏


XXX's and YYY's

There are a bunch of places in the paper with either XXX or YYY as place holders for information.
@willgray @jovo @gkiar

No ZZZ though I hope
On Wednesday, May 25, 2016, Jesse Leigh Patsolic<EMAIL_ADDRESS>wrote:

There are a bunch of places in the paper with either XXX or YYY as place
holders for information.
@will https://github.com/will @jovo https://github.com/jovo @gkiar
https://github.com/gkiar
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/neurodata/ndpaper/issues/39

Sorry, wrong Will

done.  some things are encapsulated in TODOs, mostly things that will fall out of table 3

💑


Bump 3dt to include fix for cluster checks

High Level Description
What features does this change enable? What bugs does this change fix? High-Level description of how things changed.
Related Issues

DCOS_OSS- Foo the Bar so it stops Bazzing.
DCOS- Foo the Bar so it stops Bazzing.

Checklist for all PR's

[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[ ] Read the DC/OS contributing guidelines
[ ] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and Review Process
What is the review process and when will my changes land?
All PRs require 2 approvals using github's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot bump-ee

@branden @darkonie ptal
failed tests are flaky, rerunning them.

@mesosphere-mergebot label ready for review

@amitaekbote Please update the PR description. It's not clear what fix is being added with this bump.

@branden done

The Azure test timeout is tracked via https://jira.mesosphere.com/browse/DCOS_OSS-1215. Currently with BLOCKER status until we have a solution/decision (fix or mute).

@mesosphere-mergebot  label ship it

@mesosphere-mergebot label ready for review

Merged as part of rename package 3dt -> dcos-diagnostics #1657


Compress(or resize) the image up to 100Kb

I've searched all issues with this feature but I'm not able to find. I want to compress any images up to 100kb and I don't mind how its quality is bad or resolution is small. Investigating several hours at online, I found this library is the best one for NodeJS. Please give me any possible idea to achieve this feature.

"I want to compress any images up to 100kb and I don't mind how its quality is bad or resolution is small"

Please can you provide (quite a bit) more information, including sample images and code, to help someone else understand what you're trying to achieve.

When I upload images to my server, I want to make sure that all image sizes are less than 100KB with compressing. To achieve this, I guess I should make images smaller or down the quality of the images but I'm not sure how I can implement this via this Sharp library.

How would you expect Sharp to do this for you? You want images to use exactly 100KB no more, no less? Any reason why? If you don't care about quality, being less shouldn't matter.
If your input image dimensions can vary and be very large, your first step would be to resize to a more acceptable width/height size. You've not indicated how these images are intended to be used, if they're for display usage, you should size appropriately for the dimensions they'll actually be rendered at.
After that, you could then check file size after resize, and if it's above 100KB, then you can lower the quality(eg 90, 80, 70) and repeat until the file size is at or below 100KB.
If you want to know how to use Sharp, the documentation is very nice and provides many examples for the API. You'll need to figure out what width/height you will resize images down to if needed, and add your own logic for the quality passes(checking file size is done with nodejs APIs or other package).

Thanks @polarathene I did exactly same what you said. I wanted my image to use less than 100KB(not exactly) :)


fix handling of default by = 1 when not given in nimSeq

This PR fixes a bug in default handling of a basic nimSeq case for compilation.
# In a nimbleFunction:
1:10 # works (and likely a reason this bug has not been revealed before)
nimSeq(1, 10, by = 1) # works
nimSeq(1, 10) # broke when inserting what should be default of 1.  fixed by this PR

Check why issue #1176 report is contradictory of the notes above.

All three cases look fine now. Not sure why #1176 reported that nimSeq(1, 10, by = 1) didn't work.


Unable to tap on "get your Person credential" (Person Credential Splash Screen)

Unable to click "get your Person credential" when font size is set to large

Please make all screens scrollable to avoid content behind cut below the fold.


[vm/ffi] Introduce a Pointer<T extends Struct>.refWithFinalizer(finalizer, token)

It's quite common to want to return a struct from Dart code using dart:ffi and having a call with the pointer inside that code (https://github.com/dart-lang/native/issues/1601, https://github.com/dart-lang/native/issues/1480#issuecomment-2348112178).
That currently requires the following dance:

calloc a Pointer<MyStruct>
cast the struct to <Uint8>
call asTypedList with calloc.nativeFree as finalizer
call Struct.create<MyStruct> with the typed data

Maybe we could simplify step 2 to 4 by introducing:
/// Extension on [Pointer] specialized for the type argument [Struct].
@Since('2.12')
extension StructPointer<T extends Struct> on Pointer<T> {
  /// A Dart view of the struct referenced by this pointer.
  ///
  /// Attaches [finalizer] to the backing store of the struct.
  /// The pointer (`this`) must _not_ be used anymore if the struct is _not_ guaranteed to be kept alive.
  /// Prefer doing any native calls with the pointer _before_ calling `refWithFinalizer`.
  ///
  /// Reading [ref] creates a reference accessing the fields of this struct
  /// backed by native memory at [address].
  /// The [address] must be aligned according to the struct alignment rules of
  /// the platform.
  ///
  /// Assigning to [ref] copies contents of the struct into the native memory
  /// starting at [address].
  ///
  /// This extension method must be invoked on a receiver of type `Pointer<T>`
  /// where `T` is a compile-time constant type.
  external T refWithFinalizer(Pointer<NativeFinalizerFunction>? finalizer, @Since('3.1') Pointer<Void>? token);

Hey @dcharkes , what's Pointer<Void> token for ?

See https://api.flutter.dev/flutter/dart-ffi/NativeFinalizer/attach.html documentation.

@dcharkes i am not fully understand this requirement, i understood the problem (from the comments), so can you provide a example snippet of solving the (calloc  the struct) problem with the refWithFinalizer method

  final pointer = malloc<MyStruct>();
  final myStruct = pointer.refWithFinalizer(malloc.nativeFree);

One doubt, why this being marked as external, the actual implementation needs to be written here right ? @dcharkes , btw thanks for the snippet.

@dcharkes i was almost done implementing this, one small doubt (might be a correction),
The signature you refer,
external T refWithFinalizer(Pointer<NativeFinalizerFunction>? finalizer, @Since('3.1') Pointer<Void>? token);

where the params should be named params right ? Since that is how it is for asTypedList (and also that is how this benefit "optional" params (i.e without sending null explicitly)).

@dcharkes Also am facing an issue, am doing this addition
--- a/pkg/vm/lib/modular/transformations/ffi/common.dart
+++ b/pkg/vm/lib/modular/transformations/ffi/common.dart
@@ -197,6 +197,9 @@ class FfiTransformer extends Transformer {
   final Procedure nativeEffectMethod;
+  final Procedure asUint8TypedList;
@@ -450,6 +453,11 @@ class FfiTransformer extends Transformer {
         allocatorAllocateMethod =
             index.getProcedure('dart:ffi', 'Allocator', 'allocate'),
         castMethod = index.getProcedure('dart:ffi', 'Pointer', 'cast'),
+        asUint8TypedList =
+            index.getProcedure('dart:ffi', 'Uint8Pointer', 'asTypedList'),

But while debugging using this config in vscode
    {
      "name": "dart_gen_kernel.dart",
      "type": "dart",
      "request": "launch",
      "program": "pkg/vm/bin/gen_kernel.dart",
      "args": [
        "--platform=${workspaceFolder}/out/ReleaseX64/vm_platform_strong.dill",
        "new_test.dart"
      ],
      "toolArgs": [],
      "enableAsserts": true,
      "cwd": "${workspaceFolder}"
    },
``` am not getting any errors, whereas if i once build and then run the new builded dart binary, am getting this error
`Lookup failed: Uint8Pointer|asTypedList in @methods in dart:ffi`
But it is clearly there, isn't it ?
Am i missing something to get `asTypedList` from 'Uint8Pointer' ( i did similar to how procedures of `StructPointer` and `UnionPointer` were retrieved )

Btw, to build i tried
`python3 tools/build.py -mrelease -ax64 create_sdk runtime create_platform_sdk`

Hi! It seems we had two people working on this in parallel. We've already got a working solution in https://dart-review.googlesource.com/c/sdk/+/399600. I completely missed this, sorry for the duplicate work! 🙊
One thing you could try is completely remoing out/ReleaseX64/ it might be that something is out of date in the build. Just looking that the commands it looks like you are doing the right thing.

@codesculpture I saw your earlier comments on this issue, but since some time had passed without updates and the issue wasn’t assigned, I thought maybe you weren’t working on it anymore. I should’ve checked with you first or shared the link to my CL as soon as I created it. Apologies for unintentionally duplicating the work!

@codesculpture I saw your earlier comments on this issue, but since some time had passed without updates and the issue wasn’t assigned, I thought maybe you weren’t working on it anymore. I should’ve checked with you first or shared the link to my CL as soon as I created it. Apologies for unintentionally duplicating the work!

Hey nothing to apologize, am the one who is lazy to not working on this since my previous comment, btw nice change because i compared with my change, your's is more clear !


Non-ascii characters prevent export

Bug! Both data frames have weird characters that don't want to save to any kind of text file. Must figure out how to address, and add an export function.

Fixed in this guy:
https://github.com/NeuroVault/pyneurovault/commit/887ef1cfa15f1ada8a7517db06add875da7165ca


Improve developer experience for handling server errors

There's a lot of confusion around error handling at the moment. When a server error happens, Laravel returns a non-Inertia response, which means Inertia renders the response in a modal.
People often ask for a way to disable the modal behaviour. This is not the idiomatic solution to the problem; the correct solution is to ensure your server always returns an Inertia response.
In Laravel, this can be achieved by modifying the exception handler.
A basic example:
public function render($request, Exception $exception)
{
    if ($exception instanceof ModelNotFoundException) {
        return Inertia::render('Errors/NotFound');
    }

    return parent::render($request, $exception);
}

This is a tedious job; you need to ensure you've caught all default Laravel cases, and it's a lot of boilerplate code to add to every project.
Laravel's default bahaviour is to render a Blade view based on the error's response code, like errors.404, errors.500 etc. The inertia-laravel package could mimic this method.
public function render($request, Exception $exception)
{
    return Inertia::renderException($request, $exception);
}

We could also ship a "check" for applications with multiple parts, including non-Inertia parts like an API or separate homepage.
public function render($request, Exception $exception)
{
    if (Inertia::isInertiaRequest($request)) {
        return Inertia::renderException($request, $exception);
    }

    return parent::render($request, $exception);
}

This would return an response with a standardized component name, for example Inertia::render('Errors/NotFound'). Note that JavaScript functions aren't allowed to start with a number, so we'd need to either use explicit names (NotFound), or a prefix (Error404).
What needs to be done?

First of all, this idea needs to be validated. Looking for input!
We'd need to create a mapping of Laravel exceptions to component names
There needs to be a way to specify the response code in Inertia::render

I just updated the docs with a pretty straight forward solution:
use Inertia\Inertia;

public function render($request, Exception $exception)
{
    $response = parent::render($request, $exception);

    if ($request->header('X-Inertia') && in_array($response->status(), [500, 503, 404, 403])) {
        return Inertia::render('Error', ['status' => $response->status()])
            ->toResponse($request)
            ->setStatusCode($response->status());
    }

    return $response;
}

Basically I decided it was an act of futility to try and figure out every possible exception that could get thrown. Instead, I just let Laravel generate the correct Response, and I use that decide whether I want to show an Inertia error page.
A corresponding error page component could look like this:
<template>

{{ title }}

{{ description }}

</template>

This gives developers tons of flexibility to create their own custom designed error pages.
I'm kind of happy with this as a final solution. I'm not sure this library needs to do anything more...although I'm still open to it.

I guess we could do something like this:
public function render($request, Exception $exception)
{
    return Inertia::renderErrorResponse(
        string $component,
        Response $response,
        array $codes
    );
}

For example:
public function render($request, Exception $exception)
{
    return Inertia::renderErrorResponse(
        'Error',
        parent::render($request, $exception),
        [500, 503, 404, 403]
    );
}

Again, not sure that this is worth adding to the framework, since it really doesn't require that much code to just do this yourself.

Closing this, because I think the above solution is actually perfect. 👌

I've implemented the solution outlined in the docs, but anytime I run into a 403, it's still showing laravel's stubbed 403 page... Any ideas why this might be happening?

Never mind... 🤦 I was testing locally so I got rid of the !app()->environment(['local', 'testing']) condition and I can see everything now...

Hello!
I have gotten an error when taking my app to production in a shared hosting (cpanel).
The error is that when I try to render a view, it does not render as such, but instead displays the data coming from the controller in a modal.
How can I solve this?
Thank you.


hadoop: update

[x] updated: 2.8.4->2.8.5, 2.9.1->2.9.2, 3.1.1->3.1.3
[x] removed: 3.0.x (upstream removed it too)
[x] added: 3.2.1
[x] install HDFS FUSE driver (it was built but not installed)
[x] name -> pname
[x] minor refactoring (fetched-maven-deps moved out of binary-distributon scope, sha256 renamed to sources-sha256)
[x] fix #63520 regression (https://github.com/NixOS/nixpkgs/pull/63520#pullrequestreview-288333354)
[x] native zstd and isa-l libraries
[x] fixed closure size: native libraries had link to maven.jdk in debug info (which strip -S failed to strip on fixupPhase), so it happened that hadoop closure depended on two java closures
[x] fixes #71475

cc @kamilchm @kalbasit @aespinosa

@GrahamcOfBorg build hadoop_2_7 hadoop_2_8 hadoop_2_9 hadoop_3_1 hadoop_3_2

need some work

@GrahamcOfBorg build hadoop_2_7 hadoop_2_8 hadoop_2_9 hadoop_3_1 hadoop_3_2

@GrahamcOfBorg build hadoop_2_7 hadoop_2_8 hadoop_2_9 hadoop_3_1 hadoop_3_2

@grahamc is ofborg broken?

@GrahamcOfBorg build hadoop_2_7 hadoop_2_8 hadoop_2_9 hadoop_3_1 hadoop_3_2

let's try one by one to avoid time out

@GrahamcOfBorg build hadoop_2_7

@GrahamcOfBorg build hadoop_2_8

@GrahamcOfBorg build hadoop_2_9

@GrahamcOfBorg build hadoop_3_1

@GrahamcOfBorg build hadoop_3_2

@GrahamcOfBorg build isa-l

Hashes are wrong because resolving maven artifacts always pick up the latest jodatime.
We have to

either update the hashes from time to time
or somehow pin jodatime version (it is not a direct dependency of hadoop, so it is not as easy as patching pom.xml)

I would prefer 2. because it prevents redeployment and restart

ofborg's hash mismatch of hadoop_2_10 is very strange, i cannot reproduce it :(

This is what I get:
hash mismatch in fixed-output derivation '/nix/store/cwizkwy32kygcwf7xa08ac62iz0q0154-hadoop-maven-deps-2.7.7':
  wanted: sha256:02bwcjya6mh2nxyvfb28arj7y5q42naxiyr4zikk2h7pjcxxln43
  got:    sha256:08z3im03ys4spas8idfm362b3c745ic2bn8dy4p3d7i0m3sikx2b
hash mismatch in fixed-output derivation '/nix/store/hxsdg6gag1h6sli4rpng19h36395xksj-hadoop-maven-deps-2.10.0':
  wanted: sha256:1qgdl00yrhn39z2k7pjn5q5jk275nx1mlnghsa05f94n8p0kjc3h
  got:    sha256:1787ax29dk0bsh89g4l1dixmhj9fvijw4zszmris90plp23jkcib

Seems like there is an impurity.

Some more cases:
hash mismatch in fixed-output derivation '/nix/store/1wiy91lh5zv7b55kfszzrvbam20v616j-hadoop-maven-deps-2.8.5':
  wanted: sha256:16hmzaxv52hisjc426dykls4p904sjknlmdmji5lp27pxvn0rkji
  got:    sha256:0dr2i2j9qvnkqcras553s2m97q1v9ibkym8k47f7dj1xs0v04whr
hash mismatch in fixed-output derivation '/nix/store/6kqigx2kxqj3f127l26i9wkk554bfcq0-hadoop-maven-deps-2.9.2':
  wanted: sha256:0jx51vsy1wa6mbcxfsgz07vdlwkg9n3pkp11dlpgivl0rr4krwij
  got:    sha256:0r47p9mq7pwmma47rfh5in9fa3dc79zk7vp43fvcfgmbl08r54j2
hash mismatch in fixed-output derivation '/nix/store/9wgjjmkyg9fsnwlh4qjqy92fr42cqhvx-hadoop-maven-deps-3.1.3':
  wanted: sha256:070kbphxymk3ci9794c55zrssr1sn4vrsxg07jvbwd5h4sd5hxzg
  got:    sha256:1l9x9h7baals20vn3ak686wzm0jcl3by3w7y11njcd9fc64ql1dn

@veprbl could you upload your derivations somewhere so I could compare? (unfortunately there is no option to download from ofborg) The last time it was because of jodatime upgrade, which version was pinned upstream since then (so hadoop_2_7 hash drifting is ok)

@volth Sure:
cachix use pr-68817
nix-store -r /nix/store/cwizkwy32kygcwf7xa08ac62iz0q0154-hadoop-maven-deps-2.7.7

Strangely enough I had to replace the hash again for hadoop_2_10:
hash mismatch in fixed-output derivation '/nix/store/gfxhm0a0vs9p9dj54ffakb09fl4gv32j-hadoop-maven-deps-2.10.0':
  wanted: sha256:1787ax29dk0bsh89g4l1dixmhj9fvijw4zszmris90plp23jkcib
  got:    sha256:169n1xywjx3nhwx22pcwm4s74ihpphpgkpqyafjc1b5jv34sb5sd

Here is the result
nix-store -r /nix/store/gfxhm0a0vs9p9dj54ffakb09fl4gv32j-hadoop-maven-deps-2.10.0

One problem is
https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-remote-resources-plugin/maven-metadata.xml
had <latest>1.6.0</latest> at the time of the last commit and now it is <latest>1.7.0</latest>.
Tracking and patching all those places would be boring and difficult to maintain (there are more derivations like hadoop out there)
I think, we need to froze the whole Maven Central somehow:
For example, by downloading all mutable files (like maven-metadata.xml) and putting them into a  repo on GitHub.
Or, probably, another Linux distro already has immutable snapshots of whole Maven Central?

related https://github.com/NixOS/nixpkgs/issues/25686

Whole Maven Central is only 5TB.
I think we could set up a mirror with weekly or monthly snapshots which would remain immutable.
This would make resolving process deterministic and simplify packaging of Java apps.
Immutable mirrors of NPM might have sense too.
5TB is not big, but it is bigger than GitHub limit of 100GB.
I could set up it on my premises, but it would be better to do in on servers of NixOS community (next to tarballs.nixos.org), to avoid bus factor.
@grahamc

Let's also remove hadoop 2.7 and 2.8 - they are EOL.

Let's also remove hadoop 2.7 and 2.8 - they are EOL.

Without me then - 2.7 is the only battle-tested among there derivations and the only one deployed on my premises (it is not too easy to upgrade software of such scale).

@volth I am no authority here, I was just sharing my observation when looking at the hadoop site. I do think your PR is a significant improvement over the current state and it is still worth merging. If there is a reason to keep 2.7 and 2.8 (and it seems there is in your case), I think it's OK to keep it in the nixpkgs.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/help-packaging-a-java-app/3871/13

@GrahamcOfBorg build hadoop_2_7
@GrahamcOfBorg build hadoop_2_8
@GrahamcOfBorg build hadoop_2_9
@GrahamcOfBorg build hadoop_2_10
@GrahamcOfBorg build hadoop_3_1
@GrahamcOfBorg build hadoop_3_2
@GrahamcOfBorg build hadoop_3_3
@GrahamcOfBorg build isa-l

@GrahamcOfBorg build hadoop_2_7
@GrahamcOfBorg build hadoop_2_8
@GrahamcOfBorg build hadoop_2_9
@GrahamcOfBorg build hadoop_2_10
@GrahamcOfBorg build hadoop_3_1
@GrahamcOfBorg build hadoop_3_2
@GrahamcOfBorg build hadoop_3_3
@GrahamcOfBorg build isa-l


flag label inserted into the wrong part of the entry

There is currently a problem in master where the flag label is inserted immediately after the entry tag, causing XML error and display issues in Lexonomy.
Miloš looked at this and says:

the original regexp in JS was:

.replace(/(=)"([^"]*)"/g, "$1'$2'")

the new one is:

xml = re.sub(r"(=)"([^"]*)"", "\1='\2'", xml)
notice the additional equal sign in the second argument, I guess that's a typo, or was it intentional Adam? On top of that, the second argument also needs to be double-escaped or r-string, so it should be:
xml = re.sub(r"(=)"([^"]*)"", r"\1'\2'", xml)
The regexp there is some quotes/apostrophes normalization I copied over from some other code in Lexonomy back then, I think, and it should be above all rewritten to be slightly more readable:
xml = re.sub(r"="([^"]*)"", r"='\1'", xml)

Adam replied: this seems like a typo. I vote for last regex, I'm not sure why we need = in parentheses, that was probably useful in other parts of Lexonomy, but not in this case.
We have tested this on our installation and it seems to work fine, so it can be implemented into master too.

Adam wrote: this seems like a typo. I vote for last regex, I'm not sure why we need = in parentheses, that was probably useful in other parts of Lexonomy, but not in this case.
We have now tested this on our local installation and it seems to work fine so it can be implemented into master.

Latest update: we have tested on our local installation and I noticed one weird thing - we have a tag for flags called  at this level:

So basically it is within the header info. When I select  as the flag tag and insert a tag, a new  tag is inserted immediately after the opening  tag and before . Is that expected behaviour?

Should be fixed on beta branch with commits e3c5151cfd42c770d1c41d8c128beac8e66e26e4 and be2e6cf.


Add analytics

This PR resolves https://github.com/leancodepl/patrol-internal/issues/22
package:usage doesn't support GA4. It uses the older Universal Analytics instead, which is shutting down on June 1st, 2023.
flutter_tools is using package:usage as well, but if they don't manage to upgrade this package on time, I'm pretty sure they'll get 1 year extension, which we most probably won't.


Add pre-cleanup for database tables to ensure starting fresh for each controller integration test

Cherry pick of #616

Coverage remained the same at 88.515% when pulling 5c020caf604eb90da69506726cc3c48460c621f0 on tgianos:fixBuild into 94da01ee8b812662f2b238cb04abab28ca437a2d on Netflix:3.2.x.


Documentation doesn't match latest release (e.g. objective functions)

Feature (rank_xendcg) not yet released: https://github.com/microsoft/LightGBM/issues/3142
But objective is listed in docs: https://lightgbm.readthedocs.io/en/latest/Parameters.html#objective

Thanks for using LightGBM @artdgn !
Right now, our docs at that link match the master branch of this repository, which can be  ahead of the most recent release to package managers like PyPi.
Sorry for the confusion! We could probably do a better job of adding comments like "new as of version x.x.x" under newer features.
I'm going to close this issue but please comment if anything was unclear or you need  additional help.


Remove mention of AddressSanitizer support under "Install the C++ Lin…

…ux workload in Visual Studio"
ASan support isn't very relevant when a user is trying to modify their VS install - I think this detail makes the page more confusing than it needs to be.

@esweet431 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@mikeblome : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Update AutoFormat to be more generic to support J-extended language like Kotlin

As we know, currently, AutoFormat doesn't support Kotlin well.
This PR is intended to update AutoFormat to be more generic to support J-extended language like Kotlin.

Since we are going to separate autoFormat by language, this becomes unnecessary. Closing out.


Postal and Mautic Perfect Installation on Same VPS

Hi, Hope u Everyone doing well. I really need your help. Kindly guide me to install Postal and Mautic Perfectly on Same VPS!

Use docker my friend

Thanks for reply dear. Can u write steps for this? It would really help alot

do you have skype

Yes I do. What's your Skype ID?

is this even possible? to have mautic and postal in the same vps?

Yes install first postal (normaly install) after install mautic whith docker + docker compose.

sorry i'm kinda new to all this.
I've already installed postal and it's running 10/10 now...
so next step is install docker
then docker compose
then mautic right?
Do you have any reference article with a step by step guide?
thanks for responding man!


Detach config

Hi!
Thanks for a great job firstly!
My little little improvments for this project:

Detach config and make it auto-creatable
Disabled loading PS profile to speed up startup

Updated as per comments above

What do you think about change powershell.exe call to cmd.exe?
time -f '%e' ash -c "wslpath \$(cmd.exe /c 'echo %USERPROFILE%' 2>/dev/null)"
/mnt/c/Users/smolkinpv
0.09

time -f '%e' ash -c "wslpath \$(powershell.exe -NoLogo -NoProfile -c 'Write-Host -NoNewline \$env:USERPROFILE')"
/mnt/c/Users/smolkinpv
0.96

as well as
time -f '%e' taskkill.exe /F /T /IM wsl-vpnkit.exe || :
SUCCESS: The process with PID 9072 (child process of PID 10816) has been terminated.
0.42
time -f '%e' powershell.exe -NoProfile -c 'Stop-Process -Force -Name wsl-vpnkit -ErrorAction SilentlyContinue'
0.97

@psmolkin Thanks, those improvements do seem worthwhile too if you want to open a separate PR.


Make paste automatically activated by default.

Hello, Maccy does not paste, what might be the problem and how can I fix it?
installation: brew
Current version: [0.26.2,17]
macOS version: 13.1
keyboard layout: us

https://user-images.githubusercontent.com/44067394/227779654-3d814f46-3403-4356-be4e-1684094e1024.mp4

nvm, the pasting was with option+return, not only return :D
Though it might make sense to enable paste automatically by default.

Duplicate of #500


0.14.0 release

0.14.0 (2024-12-19)

CLI mandatory API key authentication for CLI
api key based authentication and basic auth deprecation warning (#90)
ownership based search

coverage: 95.961%. remained the same
when pulling 3b90fd3263bc811e27f013aa5611b8c163c1a195 on release/0.14.0
into 7f3df6e5fa0f4aad1e7fda544b6617fdb0c210a4 on main.


message view: Redesign the more-less message interaction.

This commit encompasses the following changes:

Replace the [More...] link with a button titled "Show more".
Replace the [Show Less...] link with a button titled "Show less".
Add various on-hover interactions to the buttons.
In the condensed view, add fading to the bottom of the message to visually communicate that the message is truncated.

Fixes #22801.
CZO design discussion: https://chat.zulip.org/#narrow/stream/101-design/topic/UI.20redesign.3A.20show.20more.2Fless
Continuation of #25119 which has now been closed.

screencaptures
highlight for group and plain messages
light mode:

dark mode:

highlight for direct mention
light mode:

dark mode:

focused with unread marker showing

from previous PR, replying to Vlad's comments:
Masking gradient under more button
Asked on CZO
Hover message state
Removed that
Tuning of button styles
messagebox hover -- Done in previous section. Also just as an FYI hsla(240, 44%, 56%, 0) isn't CSS we would write -- instead of opacity 0, we'd say background transparent or white. I just removed the style for the hover change.
message_length_toggle hover, active, smallcaps, transition -- done :) I didn't know about small caps CSS, thanks!
Strange behavior
thanks for the report! There's a comparable (but not quite so bad) bug on CZO as well. I'll look into this more
I don't know how to push back to the same PR, so I opened #25787

@evykassirer works well! Left some comments.

Now updated addressing Aman's comments, with added screencaptures in the PR description for Vlad

LGTM

@terpimost let me know once you've given this your review and I'll mark it for Tim once we're done with design review :)

@evykassirer  I suggest to remove transition for the button and have active state scale to be 0.98. The rest looks good to me

@terpimost changed those two things! Tagging for integration review now.

Updated the linear gradient to 60px instead of 100px

I fixed the /help/ article being out of date and marked this to merge once CI passes, thanks for all the work on this @evykassirer, @iiishitaaaa, and @terpimost!
Also opened https://github.com/zulip/zulip/issues/26346 for a bug noticed while reviewing this.


Ungroup Mustache from HTML

Ungroups Mustache from HTML for consistency with all other templating languages per #4979.
Set colour to black as that is the only color used in the logo (site).

Set colour to black as that is the only color used in the logo (site).

Try #724b3b instead. I'm not sure I could that a logo—it's merely the Georgia font's } glyph rotated 90º with CSS  and presumably made #000 to replace the #333 inherited from the <body> element:

(I could ramble away about Gestalt's importance in graphic design, or how contrast is the device "black" logos use to establish visual identity, but I think you get the point…)

Where is #724b3b taken from?

Where is #724b3b taken from?

Google images.  I did adjust the saturation and intensity so it was more vibrant…)


RSI strategy

Usually we define buying and selling RSI strategy as 20 and 80, what are the reasons for using 30 and 70 here?

Good question! What you asked is totally the problem we want to solve by implementing Tagenalgo. In practice, traders usually modify the parameters set for different markets with same strategy due to the variety of volatility in different markets.
For example, in the high volatility markets such as foreign exchanges or cryptocurrency market, traders might set a wider range for the low and up threshold such as 20 and 80, because the price movement might maintain the same direction when the RSI value reaches 30 or 70. If so, we cannot expect to earn with the mean-reversion strategy. On the other hand, in stock market, 30 and 70 for buying and selling might be better, because the price will immediately reverse when the RSI value cross over our thresholds setting.
Though the logic sounds reasonable, we still do not know if the parameters set such as (30, 70) or (20, 80) can bring us the most profits. Therefore, we can use the genetic algorithm to find out the best local parameters set for the strategy.

Usually we define buying and selling RSI strategy as 20 and 80, what are the reasons for using 30 and 70 here?

Good question! What you asked is totally the problem we want to solve by implementing Tagenalgo. In practice, traders usually modify the parameters set for different markets with same strategy due to the variety of volatility in different markets.
For example, in the high volatility markets such as foreign exchanges or cryptocurrency market, traders might set a wider range for the low and up threshold such as 20 and 80, because the price movement might maintain the same direction when the RSI value reaches 30 or 70. If so, we cannot expect to earn with the mean-reversion strategy. On the other hand, in stock market, 30 and 70 for buying and selling might be better, because the price will immediately reverse when the RSI value cross over our thresholds setting.
Though the logic sounds reasonable, we still do not know if the parameters set such as (30, 70) or (20, 80) can bring us the most profits. Therefore, we can use the genetic algorithm to find out the best local parameters set for the strategy.

Usually we define buying and selling RSI strategy as 20 and 80, what are the reasons for using 30 and 70 here?

Good question! What you asked is totally the problem we want to solve by implementing Tagenalgo. In practice, traders usually modify the parameters set for different markets with same strategy due to the variety of volatility in different markets.
For example, in the high volatility markets such as foreign exchanges or cryptocurrency market, traders might set a wider range for the low and up threshold such as 20 and 80, because the price movement might maintain the same direction when the RSI value reaches 30 or 70. If so, we cannot expect to earn with the mean-reversion strategy. On the other hand, in stock market, 30 and 70 for buying and selling might be better, because the price will immediately reverse when the RSI value cross over our thresholds setting.
Though the logic sounds reasonable, we still do not know if the parameters set such as (30, 70) or (20, 80) can bring us the most profits. Therefore, we can use the genetic algorithm to find out the best local parameters set for the strategy.


import / require xit

Is there a way to import / require to satisfy my tooling?
I can do
import { it } from 'mocha';

but not
import { xit } from 'mocha';

or
import { xit } from 'mocha/lib/interfaces/bdd';

Looks like the x cases are missing from https://github.com/mochajs/mocha/blob/2bc9c4d1d638c8a95250757dd23724f788918abe/lib/mocha.js#L200-L214 They should be pretty easy to add, although I'll have to look around a bit more to find where we have tests for those exports so we can add tests for the x cases.

Hey @ScottFreeCode I would like to give a try to this.

We'd be happy to take a pull request!
Don't worry about including tests at this point -- I just went looking for them and it seems that we're currently only testing describe and it out of all those functions: https://github.com/mochajs/mocha/blob/f20de56637b2223f614ce40adc0d41a58030f042/test/unit/required-tokens.spec.js So we'll want to expand that, but not just for the couple of new additions, and can/should do it in a separate PR.

(Created #2982 for adding a more complete test of this feature.)

Hey @ScottFreeCode made a PR for this issue https://github.com/mochajs/mocha/pull/2997

I think this fix introduced a bug https://github.com/mochajs/mocha/issues/3004


CNDB-11283 main: Unify CQLTester#waitForIndex and SAITester#waitForIndexQueryable

Port of CASSANDRA-18521, fixes the flakiness of LiteralOrderTest.testTextPrefixes.

Thanks for the review. I have pushed a couple of fixes and CI doesn't show any regressions: http://butler-stargazer.aws.dsinternal.org/#/ci/upstream/compare/ds-cassandra-pr-gate/CNDB-11283-main


Upgrade Pillow

Update library to latest version
Fix imports and implementations related to the library
In case the library is no longer maintained, replace with an alternative and update all implementations
Alternatively update the library itself to host as another open source solution

pip install --upgrade Pillow
to upgrade the version of your pillow in django.


Any chance for vertical & horizontal options with responsive steps on the wizard?

It would be great to have a prop called something like
stepsOrientation: 'vertical|horizontal'
If you have more than 5 steps then it would be great to have the steps vertically.
Thanks again for this amazing plugin!

Nice Component.. and worked !
Any chance to make it responsive?

@pimboden the wizard its already responsive but in very small screen like phones it needs a little workout but not too much make it work smoothly. It just need to draw the wizard step line vertically and connect the icons and that should be all we need. Did you see the behavior of this wizard on small screens? Icons still get arranged perfectly but the step line still going horizontally.

Yes, you are right. this looks pretty good most of the times. The issue is if (like in my current use case) you have steps, that have e bit longer names, then the names overlap and are unreadable

@pimboden You should try to use concise names on the wizard titles to work around that. I myself sometime need to do this types of workaround to make it look OK. In the meantime we should try to apply some custom css styling to force the wizard icons to split themselves to one per line and align them on the center of the screen. If I find any workaround to make it look good on mobile devices I will paste it here.

OK thanks. The problem of the name of the steps, is that they are given by some other Autors.  I'm also trying to tweak around with css.

the wizard now has a responsive treatment. going to close this


Error message in console

Describe the bug
When creating transform using dashboards seeing the following error in console
Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-75XtnrpDA0UHDMcl7S8lvswryIOd0RqgacRh0AMOgdk='), or a nonce ('nonce-...') is required to enable inline execution.

https://github.com/elastic/kibana/issues/30468


Perform Catchup jobs chronologically

Currently, the Catchup mechanism works by requesting missing blocks in chunks, backwards from some disconnected block we want to connect to our local Transition Frontier. This is a natural way to traverse the chain since it is the direction of the block pointers, however, it would be better if we were to perform the Catchup jobs in chronological order and add them to the Transition Frontier in chunks as we go. We will need to be sure to be careful in how we design the Breadcrumb Builder under this scheme since the Breadcrumb Builder will now need to be aware of still ongoing interdependent jobs in a similar way to how the Catchup Scheduler tracks dependencies.
This is a mild improvement on the daemon, so it is not a terribly high priority. The main effect of doing this would be to allow nodes to remaining more up to date with the the Transition Frontier of others on the network while there are still ongoing Catchup jobs.

Some more detail here:
The catchup scheduler is responsible for fetching the dependencies of new
blocks. If we receive a block and don't already have its parent block stored
in the frontier, the catchup scheduler starts "watching" that block (watch
function). This includes setting a timeout. If the timeout fires before the
parent block is learned, it triggers a "catchup job". The catchup scheduler
collects all of the watched blocks in a map from parent hash -> list of
direct children (collected_transitions field), to try and avoid triggering
extra catchup jobs unnecessarily. The transition processor calls the notify
function for every transition inserted into the frontier, at which point the
catchup scheduler will look to see if the new transition satisfies the
dependencies of the watched blocks, and if it does, sends them off to the
breadcrumb builder. If the timeout fires, Ledger_catchup is used to download
the missing transitions from other nodes, and those get sent to the breadcrumb
builder.
The breadcrumb builder is responsible for applying the "staged ledger diffs",
a very expensive operation, in the background to produce a full "breadcrumb".
A breadcrumb is an "unpacked transition". Transitions include a diff from the
previous state to the state they represent, and the breadcrumb includes the
result of applying that diff. A "initial validated" transition can have a staged
ledger diff that will fail to apply, but we can't know that until we try and
apply it and we can't apply it until we have the parent state. So the breadcrumb
builder takes in a tree of transitions and produces the corresponding tree of
breadcrumbs, under the assumption that the parent state of the root of the
tree is available in the transition frontier.
The ledger_catchup module takes as input a list of trees (a "forest") of
all the relevant collected transitions. Each tree has the missing transition
as the parent of its root the transition. It downloads the state hashes of the
transitions connecting back to the root of the frontier (or the start of the root
history? I'm not exactly sure. A bunch of state hashes for sure) and then uses
those state hashes as identifiers to download the missing transitions. Once
all of the transitions have been downloaded, the completed tree of initially
validated transitions get sent off to the breadcrumb builder.
Closing this issue will require slightly changing the structure of how this
communication works. Instead of sending complete trees around, once ledger_catchup
takes over it should write each available transition immediately.
There is a complication: if the breadcrumb builder fails to apply a diff, we
should cancel all the other work happening to download any distance children of that transition.

A couple of extra details to note:

The transition processor calls the notify function for every transition inserted into the frontier, at which point the catchup scheduler will look to see if the new transition satisfies the dependencies of the watched blocks, and if it does, sends them off to the breadcrumb builder.
It also takes into account the case that we have received a dependency that does not fully connect our subtree to the frontier, and does so in a concatenative way. More concretely, take the chain A -> B -> C -> {D, E}. Assume our frontier has A, and we have active catchup jobs for D and E. If we receive C, we will remove the catchup waits for D and E and put a new wait on C. When we fire off the job for C (if we do not receive B in time), we will then fire a job to catchup to the entire subtree of C -> {D, E}.

So the breadcrumb builder takes in a tree of transitions and produces the corresponding tree of breadcrumbs, under the assumption that the parent state of the root of the tree is available in the transition frontier.
Important to note here that is submits this tree all at once to the processor rather than submitting each breadcrumb one at a time. This keeps the pressure on the processor lower, allowing it to handle network blocks more regularly without significant slowdown during catchup.

It downloads the state hashes of the transitions connecting back to the root of the frontier (or the start of the root history? I'm not exactly sure. A bunch of state hashes for sure) and then uses those state hashes as identifiers to download the missing transitions.
Currently, it just always downloads K state hashes. This is inefficient and should be changed as part of this chronological work, but it guarantees that we only every need to make one request. The missing transitions are then downloaded in chunks of a configurable size.

Closing this issue will require slightly changing the structure of how this
communication works. Instead of sending complete trees around, once ledger_catchup takes over it should write each available transition immediately.
In order to keep pressure on the processor low during catchup, I think it needs to write chunks of subtrees at once, rather than one breadcrumb at a time. This also fits into the context that we download the missing blocks in chunks as well, so it may not be that difficult, though it will require a change to how we download missing blocks. (perhaps downloading missing blocks by providing a list of state hashes makes the most sense here so that the client can determine the ordering).

There is a complication: if the breadcrumb builder fails to apply a diff, we
should cancel all the other work happening to download any distance children of that transition.
We can leverage the Unprocessed_transition_cache here. It can provide subscriptions to whether or not blocks have been determined to be invalid. Sorry, I forgot about this last we spoke on this topic.

Changes are now extending to breadcrumb builder.

@yutongz5  to setup a discussion about the approach with the team


Add endpoint for checking if file exists

Added endpoint for checking if file is present or not. #19

Looks like the test runner is failing for some random reason. I created an issue for that and I'm working on it now.

Thank you @SudhanshuJoshi09 ! This helped a lot!


[requires.io] dependency update on master branch

This change is

Current coverage is 97.23% (diff: 100%)

Merging #62 into master will not change coverage

@@             master        #62   diff @@
==========================================
  Files            13         13
  Lines           796        796
  Methods           0          0
  Messages          0          0
  Branches         42         42
==========================================
  Hits            774        774
  Misses           17         17
  Partials          5          5

Powered by Codecov. Last update 2f13b2a...1234922

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


_revinclude results need entry.search.mode "include"

Currently they have search.mode of match.
E.g. http://fhirtest.uhn.ca/baseDstu2/Patient?_id=3633&_revinclude=Condition:patient
returns entries like the following.
     {
            "resource":{
                "resourceType":"Condition",
                "id":"2511",
                "meta":{
                    "versionId":"2",
                    "lastUpdated":"2015-07-16T15:57:24.091-04:00"
                },
                "language":[
                    "3Vf9lD4km0UBj68K2mT1"
                ],
                "text":{
                    "status":"generated",
                    "div":"
Ischemic stroke, July 18, 2010
"
                },
                "patient":{
                    "reference":"Patient/3633"
                },
                "code":{
                    "coding":[
                        {
                            "system":"http://snomed.info/sct",
                            "code":"422504002",
                            "display":"Ischemic stroke (disorder)"
                        }
                    ],
                    "text":"Stroke"
                },
                "category":{
                    "coding":[
                        {
                            "system":"http://hl7.org/fhir/condition-category",
                            "code":"diagnosis",
                            "display":"Diagnosis"
                        }
                    ]
                },
                "clinicalStatus":"entered-in-error",
                "onsetDateTime":"2010-07-18",
                "notes":"RjY0_NVpcaEW2R7QNeLgWw"
            },
            "search":{
                "mode":"match"
            }
        },

Thanks for reporting! Will be committing a fix shortly, and deploying it to the test server later today.


New version: QWTWPlot v1.5.8

Registering package: QWTWPlot
Repository: https://github.com/ig-or/QWTWPlot.jl
Created by: @ig-or
Version: v1.5.8
Commit: 9ddeda81d39779e40d2ae66e1e1b3fed89df5a7e
Git reference: HEAD
Description: JULIA package for 2D/3D plotting
Release notes:

qwtw jll   updated

[merge approved]


New readme, with new code and markdown

Please refactor this PR for not include all code in the README