Add files via upload

Description
Please include a summary of the change and which issue is fixed. List any dependencies that are required for this change.
Fixes #(issue_no)
Replace issue_no with the issue number which is fixed in this PR
Have you read the Contributing Guidelines on Pull Requests?

[x] Yes
[ ] No

Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md
[x] My documentation follows Template for README.md
[x] My changes generate no new warnings
[ ] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works.

@STARK-yv What is this?

@STARK-yv What is this?

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

I got your point
But i request you to check my JARVIS repo on my profile and believe me i was not aware of the license file I used to add it because my friend suggested it

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

I got your point
But i request you to check my JARVIS repo on my profile and believe me i was not aware of the license file I used to add it because my friend suggested it

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

I got your point
But i request you to check my JARVIS repo on my profile and believe me i was not aware of the license file I used to add it because my friend suggested it

And Here https://github.com/Arbazkhan4712/JARVIS-Udemy-Code Found the Original Repo. You can even cloned the ReadMe with his stats..

@STARK-yv What is this?
Cloning someone else's repo and pushing it to my repo..is it supposed to be any kind of your contribution?

It's just an voice assistant program in python.

It's not even completed yet but I'm trying and I saw this repository and tried to contribute a simple voice assistant project so that i can understand which things must me improved and can be changed, but dude.... Although your repository is nice but that doesn't mean you can be rude

I'm not being rude, I'm telling you stop copying and pasting someone else's code it would take you no where and it would be worth nothing...Now you'd like to know how I know it's not your repo that's cause you have pushed MIT License along with your code which clearly states that it does not belong to you. Even if you copying and pasting at least hav a look at the files..

Yes I added MIT License because my friend suggested me to add this in every repo that's why I added it

It's not the License I'm Talking about it the name in the License
.
Do you still have something to say about it... and you don't push License along with the PR..

I got your point
But i request you to check my JARVIS repo on my profile and believe me i was not aware of the license file I used to add it because my friend suggested it

And Here https://github.com/Arbazkhan4712/JARVIS-Udemy-Code Found the Original Repo. You can even cloned the ReadMe with his stats..

Well @STARK-yv that doesn't solve anything..

Well @STARK-yv that doesn't solve anything..

Well @STARK-yv that doesn't solve anything..

Then run both the code, it might then....

Well @STARK-yv that doesn't solve anything..

Then run both the code, it might then....


Resolve ambiguous literature matches in post-processing step using manuscript context

The current literature grounding uses all available synonyms for(T/D/Dr).
This can produce clashes when the same synonym or name is defined in more than one instance of the same entity type. While we have added some heuristics to for example prioritise gene symbols over other gene names, we still have a decent number of ambiguous results in all entities.
The next query in the matches dataset (21.06) illustrates the problem:
(
    matches
    .groupBy("label", "type")
    .agg(F.countDistinct(F.col("keywordId")).alias("distinctGroundedIdCount"),
         F.collect_set(F.col("keywordId")).alias("distinctGroundedIds"),
         F.countDistinct(F.col("pmid")).alias("distinctPMIDs"))
    .sort(F.col("distinctGroundedIdCount").desc())
    .persist()
    .filter(F.col("distinctPMIDs") > 50)
    .show(20)
)

+--------+----+-----------------------+--------------------+-------------+
|   label|type|distinctGroundedIdCount| distinctGroundedIds|distinctPMIDs|
+--------+----+-----------------------+--------------------+-------------+
|     TEC|  GP|                    745|[ENSG00000280665,...|          221|
|     Tec|  GP|                    745|[ENSG00000280665,...|          711|
|      31|  GP|                     65|[ENSG00000005189,...|           61|
|      U3|  GP|                     50|[ENSG00000212598,...|          177|
|      U6|  GP|                     33|[ENSG00000283564,...|         1243|
|   Nor-1|  GP|                     28|[ENSG00000119508,...|           90|
|   nor-1|  GP|                     28|[ENSG00000119508,...|           88|
|     S-1|  GP|                     27|[ENSG00000119943,...|          192|
|       1|  GP|                     26|[ENSG00000119943,...|         3878|
|      .1|  GP|                     26|[ENSG00000119943,...|          464|
|      -1|  GP|                     26|[ENSG00000119943,...|         5996|
|     ́-1|  GP|                     26|[ENSG00000119943,...|           76|
|      -3|  GP|                     22|[ENSG00000106484,...|          976|
|      .3|  GP|                     22|[ENSG00000106484,...|           84|
|       3|  GP|                     22|[ENSG00000106484,...|         3028|
|   his-3|  GP|                     22|[ENSG00000106484,...|          119|
|rotamase|  GP|                     14|[ENSG00000122642,...|          117|
|  PPiase|  GP|                     12|[ENSG00000137168,...|           51|
|  PPIase|  GP|                     12|[ENSG00000137168,...|         1208|
|     p55|  GP|                     10|[ENSG00000067182,...|         2916|
+--------+----+-----------------------+--------------------+-------------+
only showing top 20 rows

Note: Don't worry about the TECs they will be fixed once the new target object kicks in. Ensembl adds TEC as a placeholder synonym and causes issues on our side.
p55 is a good example on how, a label that was used to refer to proteins with a molecular weight of 55kDa is still been used in by 2916 publications to refer to multiple proteins. Hopefully, the same manuscripts will have grounded the right entity within the 10 possibilities.
We want to design a prototype script (e.g. pyspark) that inputs the matches dataset and produces an output matches dataset in which incorrect ambiguous results have been excluded.
The idea to explore, is whether other matches in the same sentence, section or manuscript can disambiguate the correct match. For example, is expected that when a given acronym is used in the manuscript, the long name is also available somewhere in the text. By using the context we expect to resolve some ambiguous matches in a magnitude still to be determined.
We are aiming this prototype to eventually be integrated in the literature pipeline in Scala.

Some of the ambiguously grounded disease terms:
+------------------------+-----------------------------------------------------------+-----+------------------------------------------------------------------------------------------------------+
|label                   |keywords                                                   |count|labels                                                                                |
+------------------------+-----------------------------------------------------------+-----+------------------------------------------------------------------------------------------------------+
|ACE                     |[Orphanet_946, Orphanet_36, EFO_0007129]                   |3    |[Acrocephalosyndactyly, Acrocallosal syndrome, acute chest syndrome]                                  |
|ACS                     |[Orphanet_946, Orphanet_36, EFO_0007129]                   |3    |[Acrocephalosyndactyly, Acrocallosal syndrome, acute chest syndrome]                                  |
|APE                     |[EFO_0004533, MONDO_0017278, Orphanet_320]                 |3    |[alkaline phosphatase measurement, autoimmune polyendocrinopathy, Apparent mineralocorticoid excess]  |
|CA                      |[Orphanet_209908, EFO_0009860, EFO_0000311]                |3    |[Childhood apraxia of speech, chromosomal aberration frequency, cancer]                               |
|CAS                     |[Orphanet_209908, EFO_0009860, EFO_0000311]                |3    |[Childhood apraxia of speech, chromosomal aberration frequency, cancer]                               |
|CHS                     |[Orphanet_2686, Orphanet_167, EFO_0009205]                 |3    |[Cyclic neutropenia, Chédiak-Higashi syndrome, Corpuscular Hemoglobin Content]                        |
|Colon cancer            |[EFO_0000365, MONDO_0021063, EFO_1001950, EFO_0004288]     |4    |[colorectal adenocarcinoma, malignant colon neoplasm, colon carcinoma, colonic neoplasm]              |
|DM                      |[EFO_0010133, EFO_0000398, EFO_0000400]                    |3    |[diabetic maculopathy, dermatomyositis, diabetes mellitus]                                            |
|ES                      |[EFO_0000174, MONDO_0017387, EFO_0009200]                  |3    |[Ewings sarcoma, epithelioid sarcoma, Eisenmenger syndrome]                                           |
|HP                      |[EFO_1000299, MONDO_0015540, Orphanet_79430, MONDO_0017853]|4    |[Hyperplastic Polyp, hemophagocytic syndrome, Hermansky-Pudlak syndrome, hypersensitivity pneumonitis]|
|Hp                      |[EFO_1000299, MONDO_0015540, Orphanet_79430, MONDO_0017853]|4    |[Hyperplastic Polyp, hemophagocytic syndrome, Hermansky-Pudlak syndrome, hypersensitivity pneumonitis]|
|MA                      |[Orphanet_562, EFO_1001806, EFO_1001037]                   |3    |[McCune-Albright syndrome, macrophage activation syndrome, meconium aspiration syndrome]              |
|MÃ                      |[Orphanet_562, EFO_1001806, EFO_1001037]                   |3    |[McCune-Albright syndrome, macrophage activation syndrome, meconium aspiration syndrome]              |
|MDS                     |[EFO_0000198, Orphanet_531, Orphanet_565]                  |3    |[myelodysplastic syndrome, Miller-Dieker syndrome, Menkes disease]                                    |
|Non-melanoma Skin Cancer|[EFO_0009260, MONDO_0002898, EFO_0009259]                  |3    |[non-melanoma skin carcinoma, skin cancer, skin carcinoma]                                            |
|Oral cancer             |[MONDO_0023644, EFO_0003868, EFO_0003871]                  |3    |[lip and oral cavity carcinoma, mouth neoplasm, tongue neoplasm]                                      |
|Pancreatic Cancer       |[EFO_0002618, EFO_1000359, EFO_0003860]                    |3    |[pancreatic carcinoma, Malignant Pancreatic Neoplasm, pancreatic neoplasm]                            |
|Pancreatic cancer       |[EFO_0002618, EFO_1000359, EFO_0003860]                    |3    |[pancreatic carcinoma, Malignant Pancreatic Neoplasm, pancreatic neoplasm]                            |
|Pinealomas              |[MONDO_0021232, MONDO_0003249, EFO_1000475]                |3    |[pineal body neoplasm, pineal gland cancer, Pineoblastoma]                                            |
|Testicular cancer       |[EFO_0005088, EFO_0004281, MONDO_0003510]                  |3    |[testicular carcinoma, testicular neoplasm, malignant testicular germ cell tumor]                     |
+------------------------+-----------------------------------------------------------+-----+------------------------------------------------------------------------------------------------------+

(
    spark.read.parquet('gs://open-targets-pre-data-releases/21.06.5/output/literature/parquet/matches/')
    .limit(100_000)
    .filter(
        (col('type') == 'DS')
        & (col('ismapped') == True)
    )
    .select(col('label'), col('keywordId'))
    .distinct()
    .groupby(col('label'))
    .agg(
        collect_set(col('keywordId')).alias('keywords'),
        count(col('keywordId')).alias('count')
    )
    .filter(col('count') >= 3)
    .limit(20)
    .withColumn('labels', lool_up_ols(col('keywords')))
    .show(truncate=False)
)

Comparison of the old vs new target synonyms can now be done in main vs production. This is just one example illustrating how many more synonyms we have and how necessary this work will be to prevent issues.

It has been implemented.


Programmatically accessing the order of SelectStatement?

I'm trying to write a generic "cursor based pagination" helper for diesel. I'm kind of new to Rust/Diesel so I'm not even sure if what I'm trying to achieve is possible. Basically, I want to extract "cursors" for each rows, which would consist of the columns that were ORDER BY.  For example, in SELECT * FROM users ORDER BY city, id, the cursor of a row would be its (city, id) columns.
I pasted the code I have so far (a lot of which was taken from crates.io pagination helper). See the TODO for where I'm stuck. I guess I need to add some bounds to T but I'm not sure which to use and how to access the struct containing the ordered by field.
use diesel::pg::Pg;
use diesel::prelude::*;
use diesel::query_builder::*;
use diesel::query_dsl::LoadQuery;
use diesel::sql_types::{BigInt, Text};
use indexmap::IndexMap;
pub type AppResult<T> = Result<T, Box<dyn std::error::Error>>;

#[derive(QueryableByName, Queryable, Debug)]
pub struct WithCount<T> {
    #[diesel(embed)]
    pub record: T,
    #[sql_type = "::diesel::sql_types::BigInt"]
    pub total: i64,
}

pub trait WithCountExtension<T> {
    fn records_and_total(self) -> (Vec<T>, i64);
}

impl<T> WithCountExtension<T> for Vec<WithCount<T>> {
    fn records_and_total(self) -> (Vec<T>, i64) {
        let cnt = self.get(0).map(|row| row.total).unwrap_or(0);
        let vec = self.into_iter().map(|row| row.record).collect();
        (vec, cnt)
    }
}

#[derive(Debug, Clone, Copy)]
pub struct PaginationOptions<'a> {
    after: Option<&'a String>,
    pub per_page: u32,
}

impl<'a> PaginationOptions<'a> {
    pub fn new(params: &'a IndexMap<String, String>) -> AppResult<Self> {
        const DEFAULT_PER_PAGE: u32 = 10;
        const MAX_PER_PAGE: u32 = 100;

        let per_page = params
            .get("per_page")
            .map(|s| s.parse().map_err(|e| "per page error"))
            .unwrap_or(Ok(DEFAULT_PER_PAGE))?;

        if per_page > MAX_PER_PAGE {
            return Err(format!("cannot request more than {} items", MAX_PER_PAGE,).into());
        }

        Ok(Self {
            after: params.get("after"),
            per_page,
        })
    }

    pub fn after(&self) -> Option<&String> {
        self.after
    }
}

pub trait Paginate: Sized {
    fn paginate(self, params: &IndexMap<String, String>) -> AppResult<PaginatedQuery<Self>> {
        Ok(PaginatedQuery {
            query: self,
            options: PaginationOptions::new(params)?,
        })
    }
}

impl<T> Paginate for T {}

#[derive(Debug)]
pub struct Paginated<'a, T> {
    records_and_total: Vec<WithCount<T>>,
    options: PaginationOptions<'a>,
}

impl<'a, T> Paginated<'a, T> {
    pub fn total(&self) -> Option<i64> {
        Some(
            self.records_and_total
                .get(0)
                .map(|row| row.total)
                .unwrap_or_default(),
        )
    }
    pub fn iter(&self) -> impl Iterator<Item = &T> {
        self.records_and_total.iter().map(|row| &row.record)
    }
}

impl<'a, T: 'static> IntoIterator for Paginated<'a, T> {
    type IntoIter = Box<dyn Iterator<Item = Self::Item>>;
    type Item = T;

    fn into_iter(self) -> Self::IntoIter {
        Box::new(self.records_and_total.into_iter().map(|row| row.record))
    }
}

#[derive(Debug)]
pub struct PaginatedQuery<'a, T> {
    query: T,
    options: PaginationOptions<'a>,
}

impl<'a, T> PaginatedQuery<'a, T> {
    pub fn load<U>(self, conn: &'a PgConnection) -> QueryResult<Paginated<U>>
    where
        Self: LoadQuery<PgConnection, WithCount<U>>,
        T: std::fmt::Debug,
    {
        let options = self.options;
        // dbg!(&self.query.order); // no field `order` on type T
        // TODO: extract the ORDER BY clause
        let records_and_total = self.internal_load(conn)?;
        Ok(Paginated {
            records_and_total,
            options,
        })
    }
}

impl<'a, T> QueryId for PaginatedQuery<'a, T> {
    const HAS_STATIC_QUERY_ID: bool = false;
    type QueryId = ();
}

impl<'a, T: Query> Query for PaginatedQuery<'a, T> {
    type SqlType = (T::SqlType, BigInt);
}

impl<'a, T, DB> RunQueryDsl<DB> for PaginatedQuery<'a, T> {}

impl<'a, T> QueryFragment<Pg> for PaginatedQuery<'a, T>
where
    T: QueryFragment<Pg>,
{
    fn walk_ast(&self, mut out: AstPass<'_, Pg>) -> QueryResult<()> {
        out.push_sql("SELECT * FROM (");
        out.push_sql("SELECT *, COUNT(*) OVER () FROM (");
        self.query.walk_ast(out.reborrow())?;
        out.push_sql(") t ");
        out.push_sql(") s ");
        if let Some(after) = self.options.after() {
            out.push_sql(" WHERE username < ");
            out.push_bind_param::<Text, _>(&after)?;
        }
        out.push_sql(" LIMIT ");
        // TODO: per_page + 1.. look ahead one row.. if that row not returned means has_next_page =
        // false
        out.push_bind_param::<BigInt, _>(&i64::from(self.options.per_page))?;
        /*
        if let Some(after) = self.options.after() {
            out.push_sql(" WHERE username < ");
            out.push_bind_param::<Text, _>(&after)?;
        }
        */
        Ok(())
    }
}

#[cfg(test)]
mod tests {

    use indexmap::IndexMap;

    use super::*;
    use diesel::prelude::Connection;
    use std::env;

    pub fn establish_connection() -> PgConnection {
        let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
        PgConnection::establish(&database_url)
            .expect(&format!("Error connecting to {}", database_url))
    }

    #[derive(Queryable, Debug, Clone, PartialEq)]
    pub struct User {
        pub username: String,
        pub city: String,
        pub is_suspended: bool,
    }

    #[test]
    #[ignore]
    // Important:
    // For cursor based pagination to work:
    // 1) the query must be ordered by at least one unique column the last column in the ORDER BY
    //    clause).
    // 2) The sorted columns must be returned by the SELECT so that it's possible to form cursors
    #[test]
    fn cursor_paginate_no_after() -> AppResult<()> {
        use crate::schema::users::dsl::*;
        let conn = establish_connection();
        let query = users
            .select((username, city, is_suspended))
            .order_by((city.desc(), username.asc()))
            .filter(is_suspended.eq(false));
        let mut params = IndexMap::new();
        params.insert(String::from("per_page"), String::from("3"));

        let res = query.paginate(&params)?.load::<User>(&conn);
        dbg!(&res);

        Ok(())
    }
}

First of all this is not really a topic for our issue tracker, as we try to use the issue tracker for tracking bugs and feature requests, not for answering support questions. Therefore I will close this "issue" as it does not really contain something actionable for a potential contributor to work on. The right place to ask such questions are either our gitter channel or or forum.
To answer the asked question shortly: You cannot access the field of a concrete type (SelectStatement) in a generic context. All rustc knows at the location of your TODO is that there is some type T that implements Debug, that does not allow you to access any details of a concrete type there. This is a general rust restriction and nothing diesel related. (Beside of that, we do not consider the existence of SelectStatement to be part of our public API, so accessing a field of that type is clearly outside of what is currently supported by diesel.)

Thanks, created a topic here: https://discourse.diesel.rs/t/cursor-based-pagination-with-diesel/149


chore: increase versions

Coverage remained the same at 97.143% when pulling a0401c9fc5d7c38d4817f518d4fb11703a53bc84 on chore/travis into ee8190f1f070cab7bb36d26b3004424e55abec22 on master.


Fix 30388

Purpose

improving the return statement added in a function that returns nil

For
function func() {
     re<cursor>
}

this adds return; instead of return

Fixes #30388
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Important note on correct use of PowerShell Module

I have created a change where we need to be clearer on which PowerShell module we should use on this kind of checking. It is normal that users use the SCC powershell to see the logs where in the reality we must use the EXO. This was discussed in one ICM opened.

@rtmorato PR merged. I revised the note and move it. The sentence immediately preceding the cmdlet example says to run the following command in EXO PS. So I moved the note to after the cmdlet example so as not to same the same thing twice in two consecutive sentences.

Thank you Mark.


It Doesnt allow to enable auto login after self registration from console without filling Minimum number of forced security questions to be answered

How to reproduce:

Access console as admin:admin
Go to Manage > Account Recovery
Try enabling and disabling the field auto login after self registration
Will get a UI error message as Minimum number of forced security questions to be answered a value is required for this field
But in configs I haven't enabled Security question based password recovery

When enabling this config UI should not prompt a error message as some other input is required for security questions.

https://user-images.githubusercontent.com/31848014/162128434-66c9949f-0524-4f80-9007-6ee15f0ad541.mp4
Environment information (Please complete the following information; remove  any unnecessary fields) :
5.12.0 alpha 18

Duplicates https://github.com/wso2/product-is/issues/13241


Nullable<T&>

Work in progress / Experiment / Proof of concept
This is to make the following to work:
  struct S
  {
    std::string Message;
    Azure::Nullable<S&> Next; // like a unique_ptr, but with Nullable syntax, and is byval-copyable
  };

Specializing it as T& has more of a syntactical purpose, so the type is close to T. Maybe, T&& or event T* would be better or less confusing.
Maybe even current Nulable becomes template<typename T, bool isIncomplete = false>, and this one becomes template<typename T> Nullable<T, true>

Don't look at cpp - core - ci (PrePublishBuild GenerateReleaseArtifacts) CI failing - there seems to be a problem in I at the moment, I doubt it even related to my changes. FWIW, everything compiles and tests do pass.

This was successful as a proof of concept and a demo, and at the moment I am not planning to further develop this PR, so I am closing it.
We probably can get away with using unique_ptrs, given how rarely we need to have struct S{ S? s; }.
I am thinking, if we need this, maybe we should try to make this template specialization implementation the main implementation, and remove the specialization. I.e. Nullable will become a wrapper around unique_ptr, it will be incomplete-type-friendly, internally it will be a wrapper around unique_ptr with custom deep-clone copy constructors, while 100% keeping the today's API and behavior. It might end up less confusing thatn having a special meaning for Azure::Nullable<X&>, and using both Azure::Nullable<X> and Azure::Nullable<X&> in different places in our SDK. If we make Azure::Nullable<X> incomplete-type-friendly, for struct S{ S? s; } case, we will need to put it as struct S{ Azure::Nullable<S> s; }, and that will be sufficient. struct S{ Azure::Nullable<S&> s; } is still rather confusing, i.e. you'd expect it to be basically a raw pointer without owning the object, but it is not the case.


DatePickerAndroid opens both "spinner" and "calendar" on Tablet

🐛 Bug Report
When I open DatePickerAndroid with mode "spinner" in API 28 phone (Android 8.0) it opens correctly.
But if I change to a tablet (Nexus 10 API 27 2560x1600 xhdpi), then both spinner and calendar open as you can see in the screenshot: https://monosnap.com/file/q94zFk9cjywdNFZ5fXX16eRHfhz4iO
I'm using react-native: 0.58.3
There was the same issue: "https://github.com/facebook/react-native/issues/21313"
To Reproduce
Try to run example code in any component on the tablet.
Expected Behavior
I would like to see only spinner component
Code Example
componentDidMount() {
  try {
      const {action, year, month, day} = await DatePickerAndroid.open({
        date: new Date(2020, 4, 25),
        mode: 'spinner'
      });
  }
}

Environment
React Native Environment Info:
System:
OS: macOS 10.14.2
CPU: (8) x64 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
Memory: 40.26 MB / 16.00 GB
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 8.9.4 - /usr/local/bin/node
Yarn: 1.7.0 - /usr/local/bin/yarn
npm: 6.1.0 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 12.1, macOS 10.14, tvOS 12.1, watchOS 5.1
Android SDK:
API Levels: 23, 24, 25, 26, 27, 28
Build Tools: 23.0.1, 24.0.2, 25.0.1, 25.0.2, 25.0.3, 26.0.1, 26.0.2, 26.0.3, 27.0.3, 28.0.3
System Images: android-23 | Intel x86 Atom_64, android-23 | Google APIs Intel x86 Atom_64, android-26 | Google APIs Intel x86 Atom_64, android-26 | Google Play Intel x86 Atom, android-27 | Google APIs Intel x86 Atom, android-27 | Google Play Intel x86 Atom, android-28 | Intel x86 Atom_64, android-28 | Google APIs Intel x86 Atom_64, android-28 | Google Play Intel x86 Atom_64
IDEs:
Android Studio: 3.1 AI-173.4670197
Xcode: 10.1/10B61 - /usr/bin/xcodebuild
npmPackages:
react: 16.6.0-alpha.8af6728 => 16.6.0-alpha.8af6728
react-native: 0.58.3 => 0.58.3
npmGlobalPackages:
create-react-native-app: 1.0.0
react-native-cli: 2.0.1

Hello, since this is not very active, I think you should try to fix it using these kinds of thing: https://stackoverflow.com/questions/28740657/datepicker-dialog-without-calendar-visualization-in-lollipop-spinner-mode and submit a pull request.

Thanks for your solution but nowadays I work on another project and this problem I have solved by using another library (react-native-wheel-datepicker https://www.npmjs.com/package/react-native-wheel-datepicker)


Implement RoaringBitmap.orNot operation #352

In-place orNot through a member method
non in-place orNot through a class method

Thanks for submitting these changes - it's very likely I will use this feature so want to help assure that it's correct and significantly outperforms alternatives already possible with the public API. I started looking at the changes this evening but encountered some issues running benchmarks the way I am used to. Unfortunately it will be a couple of days before I get more time to look at this.

@mdouaihy I took a look at the PR this afternoon. I refer to the evaluation code based on your branch here. I assume throughout that the implementation should satisfy the below, and compare your implementation to it.
  static void verifyInvariance(RoaringBitmap left, RoaringBitmap right, long limit) {
    RoaringBitmap actual = RoaringBitmap.orNot(left, right, limit);
    RoaringBitmap range = new RoaringBitmap();
    range.add(0, limit);
    RoaringBitmap rightFlipped = RoaringBitmap.and(right, range);
    rightFlipped.flip(0, limit);
    RoaringBitmap expected = RoaringBitmap.or(RoaringBitmap.and(left, range), rightFlipped);
    assertEquals(expected, actual);
  }

I wrote a benchmark to compare your implementation with what a user could currently do, and found this PR gives a very large performance gain (maybe even 20x).
I wrote some basic scenario based tests which all passed.
I wrote a fuzz test and it didn't take long to find cases where the new function creates empty bitmaps. These seem to be limited to cases where the last value in the bitmap is signed negative. These cases are at src/test/resources/testdata/fuzz/ornot on my branch, but more can be generated by running Fuzzer.orNot.

So I think this is very good, but it looks like there is a signedness bug or two to track down prior to merging.

@richardstartin, I fixed the code according to your suggestions.
I also integrated part of your code to enrich the Fuzzer tests. That allowed me to find some bugs, mainly at the edge of the universe (the last container at key = 65535).
I also integrated the benchmarks from your change in the PR.

@mdouaihy I left a couple more comments but this looks very good to me. I looked at the build failure and it looks like travis being temperamental. Removing the accidental import will run the build again in any case.
@lemire this looks good to me - do you want to take a look?

@richardstartin We are not porting this over to the buffer package? If it is not done within this PR, I think we should at least create a new issue for it.
From the look of it, this is a rather high level implementation, so a port to the buffer package would be easy.
Thoughts?

@lemire, I pushed an implementation in the buffer package. As you said, the port was easy since it's the same algorithm.

Waiting for @richardstartin 's "second" approval before merging.

Looks really good 👍

Merging.


Plot time relative to the start of the run

Description

The run start time is subtracted from all times in the run so that anything plotted vs time shows the time relative to the start of the run instead of seconds since 1970 (the UNIX epoch).
Motivation and Context

Closes #156.
How Has This Been Tested?
Tested interactively by plotting the results of bp.count on simulated detectors.
Screenshots:

This test failure looks related to the PR:
https://github.com/bluesky/bluesky-widgets/runs/4081964781?check_suite_focus=true#step:9:12881
Perhaps all we need to do is put namespace["time"] = ... inside a conditional that checks if "time" in namespace.


Spike: How might we support databases

What's the task?
The Next.js template doesn't currently provide any out-of-the-box database support like the Flask template. There's no reason a project couldn't use the Next.js app exclusively though, and there's been at least one instance where we've had to figure out Next.js template DB support at a project level.
The template should provide similar DB support as the Flask template, such as an ORM and DB migrations.
Prior art
The "Flex" initiative used Prisma with the Next.js template. The team ran into several pain points with Prisma, so it may not be the right solution, but there could be aspects that can serve as a useful reference for this ticket. Here are some potential relevant parts:

DB migration
DB lib

We'd need to dive in a bit more to see if it can help with the above-mentioned pain points, but I wanted to note that we should look at Drizzle ORM. I haven't yet used it myself, but everyone (online) that I have heard talk about it has said good things. Unlike the Prisma-specific schema declarations, everything is specified in TS code.
It supports a "query" syntax that is abstracted at the same level of what one would do in Prisma, but also supports an "Select" syntax where TS commands more closely resemble SQL queries for cases where performance tuning or other consideration might be helpful.


cant run menu GUI

i using matlab 2018a and the program cant run

can u help me to fix the issues?

The source code is based on some older versions of MATLAB. While the training code and the neural nett architecture can still be referenced from this repository.


add service test case - [closed]

In GitLab by @sokril on Jan 16, 2018, 08:33
Merges test -> dev

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the coverage is  % check console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
the build is FAILED console log

In GitLab by @wknet123 on Nov 13, 2019, 10:26
closed


add mean centering to clipping as default

We were getting nans with median centered clipping that seem to be at least partially resolved with mean centered clipping of the local energies. This PR allows for the choice at the command line, with the default set to mean-centered clipping.

Thanks for the review!


No image size resolving in visitor

Hi. Thanks for wonderful Markdown view, was a life-saver to me.
Here: SpannableMarkdownVisitor#visit(org.commonmark.node.Image)
The AsyncDrawable is created without imageSizeResolver and it's rendered veeeery big
I understand that there are no size constraints in native Markdown syntax but at least some level of customization should be possible for images to fit/scale into the textview.

Hey @Adonai !
I have created a new branch for the next release and pushed there discussed changes, here. I have tested it and it works for me. How it's for you? 😉

@noties , I'll test it this week and report, thanks!

Ugh, compiling aar without uploading it is such a pain.
You should have told me about jitpack earlier, bro :)
https://jitpack.io/com/github/noties/Markwon/v1.0.5-SNAPSHOT

Dafuq is this? 😶  It's not created by me.
While we are on this I can recommend using  local maven repo: https://noties.github.io/blog/2018/03/24/local-maven-repo no need to upload anything

This thing is auto-generating build artifacts for any github repo you request it to do.
See this, very handy

Yeah, I don't like it. Maybe I should create a snapshot myself as development for the next release is mainly in one branch.

Ok, tested this, works as expected. You may close this once it's released.

I think if you do something like this it should solve the RecyclerView issue:
public class MyRecyclerViewAdapter extends RecyclerView.Adapter<MyHolder> {

    @Override
    public void onBindViewHolder(@NonNull final MyHolder holder, final int position) {
        whenReady(holder.text, new Runnable() {
            @Override
            public void run() {
                Markwon.setMarkdown(holder.text, getMarkdown(position));
            }
        });
    }

    private static void whenReady(@NonNull final View view, @NonNull final Runnable action) {
        if (view.getWidth() == 0) {
            view.getViewTreeObserver().addOnPreDrawListener(new ViewTreeObserver.OnPreDrawListener() {
                @Override
                public boolean onPreDraw() {
                    if (view.getWidth() > 0) {
                        view.getViewTreeObserver().removeOnPreDrawListener(this);
                        action.run();
                    }
                    return true;
                }
            });
        } else {
            action.run();
        }
    }
}

Thanks, I solved it the other but fairly similar way.
Anyway, this issue is resolved.


Add TFR vector visualization

Porting over https://github.com/mne-tools/mne-python/pull/11524
MWE:
import numpy as np
import mne
import mne_gui_addons
from mne.datasets import somato
from mne.time_frequency import csd_tfr

data_path = somato.data_path()
subject = '01'
task = 'somato'
raw_fname = (data_path / 'sub-{}'.format(subject) / 'meg' /
             'sub-{}_task-{}_meg.fif'.format(subject, task))

# crop to 5 minutes to save memory
raw = mne.io.read_raw_fif(raw_fname).crop(0, 300)
raw.load_data()

picks = mne.pick_types(raw.info, meg='mag', eog=True, exclude='bads')

# read epochs
events = mne.find_events(raw)
epochs = mne.Epochs(raw, events, event_id=1, tmin=-1.5, tmax=2, picks=picks,
                    preload=True, decim=3)

# read forward operator and point to freesurfer subject directory
fname_fwd = (data_path / 'derivatives' / 'sub-{}'.format(subject) /
             'sub-{}_task-{}-fwd.fif'.format(subject, task))
subjects_dir = data_path / 'derivatives' / 'freesurfer' / 'subjects'

fwd = mne.read_forward_solution(fname_fwd)

rank = mne.compute_rank(epochs, tol=1e-6, tol_kind='relative')
active_win = (0.5, 1.5)
baseline_win = (-1, 0)

# compute cross-spectral density matrices
freqs = np.logspace(np.log10(12), np.log10(30), 9)

# time-frequency decomposition
epochs_tfr = mne.time_frequency.tfr_morlet(
    epochs, freqs=freqs, n_cycles=freqs / 2, return_itc=False,
    average=False, output='complex')
epochs_tfr.decimate(20)  # decimate for speed

csd = csd_tfr(epochs_tfr, tmin=-1, tmax=1.5)
baseline_csd = csd_tfr(epochs_tfr, tmin=baseline_win[0], tmax=baseline_win[1])
ers_csd = csd_tfr(epochs_tfr, tmin=active_win[0], tmax=active_win[1])

surface = subjects_dir / subject / 'bem' / 'inner_skull.surf'
vol_src = mne.setup_volume_source_space(
    subject=subject, subjects_dir=subjects_dir, surface=surface,
    pos=10, add_interpolator=False)  # just for speed!

conductivity = (0.3,)  # one layer for MEG
model = mne.make_bem_model(subject=subject, ico=3,  # just for speed
                           conductivity=conductivity,
                           subjects_dir=subjects_dir)
bem = mne.make_bem_solution(model)

trans = fwd['info']['mri_head_t']
vol_fwd = mne.make_forward_solution(
    raw.info, trans=trans, src=vol_src, bem=bem, meg=True, eeg=True,
    mindist=5.0, n_jobs=1, verbose=True)

# Compute source estimate using MNE solver
snr = 3.0
lambda2 = 1.0 / snr ** 2
method = 'MNE'  # use MNE method (could also be dSPM or sLORETA)

# make a different inverse operator for each frequency so as to properly
# whiten the sensor data
inverse_operator = list()
for freq_idx in range(epochs_tfr.freqs.size):
    # for each frequency, compute a separate covariance matrix
    baseline_cov = baseline_csd.get_data(index=freq_idx, as_cov=True)
    baseline_cov['data'] = baseline_cov['data'].real  # only normalize by real
    # then use that covariance matrix as normalization for the inverse
    # operator
    inverse_operator.append(mne.minimum_norm.make_inverse_operator(
        epochs.info, vol_fwd, baseline_cov))

# finally, compute the stcs for each epoch and frequency
stcs = mne.minimum_norm.apply_inverse_tfr_epochs(
    epochs_tfr, inverse_operator, lambda2, method=method,
    pick_ori='vector')

viewer = mne_gui_addons.view_vol_stc(
    stcs, subject=subject, subjects_dir=subjects_dir,
    src=vol_src, inst=epochs_tfr)
viewer.go_to_extreme()  # show the maximum intensity source vertex
viewer.set_cmap(vmin=0.25, vmid=0.8)
viewer.set_3d_view(azimuth=40, elevation=35, distance=300)

For now you need to black . or so. We could add this as a pre-commit hook, or add a CI to do it
For the black rendering can you try adding this line
https://github.com/mne-tools/mne-python/blob/686857c55288ed91cc13cdc43bf100ab1f6c475d/.circleci/config.yml#L103

Hmmm black . says all local files left unchanged but style still fails

Ah I think it was black installed as the debian and not the pipy distribution, just to note

Also I don't have write access FYI

@larsoner , have you solved this CI issue before?


Password File similar to pgpass

Hi OmniSci Community,
I am using the pymapd connector to successfully connect my python program to my MapD database. Currently, the user credentials are stored in my code as plain text. I was wondering if there was a tool that existed similar to a pgpass file for postgres. I would like to store my password in some sort of separate document from the code. Is this possible?
If it matters… Using Python 3.6.6 on a Windows 10 Enterprise machine. Making connection to mapd running on Rhel 7 Server.
Thanks,
Zach

Closing in lieu of discussion on the community forum:
https://community.omnisci.com/t/pymapd-password-file/1830/2


How can i set a formula into a ceil wich works on every lang ? (SUM, ROUND..)

I search a way to add into a ceil a formula like : ROUND((A1*B1);2)
If i put this line it will works fine if Excell is in English. But how can i do to modify this formula to be able to make this ROUND function for all Excell languages ?

You have to write the formula in the "English" style, with English names (e.g. ROUND) and commas to separate arguments.  The correct formula in this case is:
ROUND((A1*B1), 2)

And this should do the right thing in other languages.


[Question] Any plan to contribute this back to Kubernetes

I found that this repo actually have two code base, one is pruned kubernetes and another is k3s, two questions here:

Why host the pruned k8s and k3s code in same repo?
Any plan to contribute the pruned k8s to k8s community?

Most legacy.  k3s used to be all one code base.  It was the k8s source plus a bunch of new code.  Then we separated out to two separate packages.  Now that it's here it's hard to ever move and not break old builds.
Yes.  We are continuing to refine the set the patches on k8s so that they can be upstreaming.  The first area of focus is developing an approach to the sqlite that can be done without modifying upstream.  Most info to come there...

Yes. We are continuing to refine the set the patches on k8s so that they can be upstreaming. The first area of focus is developing an approach to the sqlite that can be done without modifying upstream. Most info to come there...

Thanks @ibuildthecloud , this is really helpful! Do you have a timeline when this can be upstreamed? ;-)

Closing due to age. K3s is moving to CNCF and we have another issue for getting rid of carried patches.


Documents for Docker-compose

Fixes #989
Changes proposed in this pull request:

resolves the conflicts between README.md and contribution guide about the process of installation
adds necessary setup instructions regarding docker-compose up.

cc @jkleinsc @jglovier @tangollama

@jkleinsc I have changed the doc according  to your  suggestion.Kindly have a look at it


Segfault when running unit tests.

(.env) sidsachdeva@ip-10-0-4-107:~/bilm-tf$ python3 -m unittest discover tests/ (...) Loaded 3 sentences. Finished loading ........2018-10-02 20:55:29.707411: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2018-10-02 20:55:29.796203: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2018-10-02 20:55:29.796619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1411] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:1e.0 totalMemory: 11.17GiB freeMemory: 11.10GiB 2018-10-02 20:55:29.796654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1490] Adding visible gpu devices: 0 2018-10-02 20:55:30.109867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-10-02 20:55:30.109930: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977]      0 2018-10-02 20:55:30.109945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0:   N 2018-10-02 20:55:30.110228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10757 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7) WARNING:tensorflow:From /home/sidsachdeva/bilm-tf/bilm/model.py:384: calling squeeze (from tensorflow.python.ops.array_ops) with squeeze_dims is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead NOT USING SKIP CONNECTIONS 2018-10-02 20:55:32.631693: E tensorflow/stream_executor/cuda/cuda_dnn.cc:343] Loaded runtime CuDNN library: 7.1.4 but source was compiled with: 7.2.1.  CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration. Segmentation fault (core dumped)
Hey when I tried to run the tests described in the readme I get this segfault. I know this is probably way lower level of an issue than you guys would be responsible for, but I wonder if you might know how to fix it? Is it because of the difference in CuDNN library?

No idea what's causing this.  Check your version of tensorflow, the code is only tested with tensorflow==1.2.

You're right, I had tensorflow-gpu==1.11.
However, now when I reinstalled the correct version and changed the, I'm getting a new error.
ImportError: Traceback (most recent call last):   File "/home/sidsachdeva/bilm-tf/.env/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>     from tensorflow.python.pywrap_tensorflow_internal import *   File "/home/sidsachdeva/bilm-tf/.env/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>     _pywrap_tensorflow_internal = swig_import_helper()   File "/home/sidsachdeva/bilm-tf/.env/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper     _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)   File "/usr/lib/python3.5/imp.py", line 242, in load_module     return load_dynamic(name, filename, file)   File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic     return _load(spec) ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory
What CUDA version are you using? I have both 8.0 and 9.0 in my path, which are installed as standard in the Amazon Deep Learning AMI

I don't the CUDA version should matter, as long as it is one supported by the particular version of tensorflow.  It seems tensorflow can't find your CUDNN library, so this is an environmental issue. I'd recommend checking the typical environmental variables, CUDA_HOME, LD_LIBRARY_PATH, etc.

Yeah so I checked those variables and they were configured as they should be. I found a fix in this issue thread: https://github.com/tensorflow/tensorflow/issues/8898 by using ln to create a link. Unfortunately that hasn't seemed to solve the segfault. I'm getting this error now when I run the tests.
........2018-10-03 00:02:42.547567: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2018-10-03 00:02:42.547611: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2018-10-03 00:02:42.547634: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. 2018-10-03 00:02:42.547657: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations. 2018-10-03 00:02:42.547680: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations. 2018-10-03 00:02:42.638421: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2018-10-03 00:02:42.638815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate (GHz) 0.8235 pciBusID 0000:00:1e.0 Total memory: 11.17GiB Free memory: 11.10GiB 2018-10-03 00:02:42.638853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 2018-10-03 00:02:42.638879: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 2018-10-03 00:02:42.638914: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0) /home/sidsachdeva/bilm-tf/.env/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py:499: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead   return np.fromstring(tensor.tensor_content, dtype=dtype).reshape(shape) NOT USING SKIP CONNECTIONS /home/sidsachdeva/bilm-tf/.env/lib/python3.5/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead   if d.decorator_argspec is not None), _inspect.getargspec(target)) 2018-10-03 00:02:45.133517: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Loaded runtime CuDNN library: 7104 (compatibility version 7100) but source was compiled with 5110 (compatibility version 5100).  If using a binary install, upgrade your CuDNN library to match.  If building from sources, make sure the library loaded at runtime matches a compatible version specified during compile configuration. 2018-10-03 00:02:45.134208: F tensorflow/core/kernels/conv_ops.cc:671] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms) Aborted (core dumped)
Though I'm able to load tensorflow and use the GPU, I'm not passing the tests for a specific reason. Any thoughts? Sorry for posting these annoying environment errors, I'm only using tf for this project because I want to train my own elmo, otherwise I usually use pytorch, where I almost never have such issues.

This could be a mismatch between your installed CUDA / CUDNN version and the one that tensorflow==1.2 expects.  The last few lines of your error message:
Loaded runtime CuDNN library: 7104 (compatibility version 7100) but source was compiled with 5110 (compatibility version 5100). If using a binary install, upgrade your CuDNN library to match. If building from sources, make sure the library loaded at runtime matches a compatible version specified during compile configuration


Added commands to {build, run} current example

Hi and thank you for contribution.
Please run gulp tslint and fix linter issues if you can :)

I have no gulp.

Then you should do following steps:
npm install -g gulp
cd /path/to/your/fork/of/RustyCode/
gulp tslint

Then you should see list of style errors which need to be fixed :)

You can also see the error list here:
https://travis-ci.org/saviorisdead/RustyCode/builds/96774554

I fixed styles


Wrong line ending on Windows

Windows loves its \r\n line endings, which makes a test fail:
1) #findAllDependencies
       works for a module with comments between lines with three dependencies:

      AssertionError: expected [ Array(2) ] to deeply equal [ Array(3) ]
      + expected - actual

       [
         "C:\\Users\\Johannes\\Code\\find-elm-dependencies\\test\\fixtures\\Test\\ChildA.elm"
         "C:\\Users\\Johannes\\Code\\find-elm-dependencies\\test\\fixtures\\Test\\ChildB.elm"
       + "C:\\Users\\Johannes\\Code\\find-elm-dependencies\\test\\fixtures\\Native\\Child.js"
       ]

By using os.EOL instead of \n, this problem can be fixed for me. This fix is applied in the PR #26 in commit 962ce6b18699584262f0b4ba869810d27dce45fc.

I'm a little bit afraid that this might fail, if for some reason a file on Windows has the \n line ending. An better fix would be to use the regex \r?\n, but I'm not sure about the performance implications and how to measure them.


Page is missing details on the share the app option

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 72ff3e88-8793-8e2b-bdb2-0e3d09372b93
Version Independent ID: 0e1245cb-d7af-c73c-39c4-39f5665ae51a
Content: Publish an app in Power BI - Power BI
Content Source: powerbi-docs/collaborate-share/service-create-distribute-apps.md
Service: powerbi
Sub-service: pbi-collaborate-share
GitHub Login: @maggiesMSFT
Microsoft Alias: maggies

@samaguire -- so sorry for my delayed response! This article has been updated many times since you entered this feedback, and now does mention the "share the app" functionality. Thanks!


Maintenance: Remove all lane connections

Resolves #1646
Simple maintenance tool for removing all lane connections. More info in #1637
@kianzarrin I'm pretty sure that resetting connection database and requesting full routing recalculation should be enough to make it work correctly. Please double-check.

does it also call the notifier to notify AN of changes?

does it also call the notifier to notify AN of changes?

Valid point. Which event it should trigger?

Valid point. Which event it should trigger?

Notifier.OnNodeModified() for every node.

Valid point. Which event it should trigger?

Notifier.OnNodeModified() for every node.
EDIT: the thing is that changes to lane connections triggers rerouting and routing manager in turn invokes the notifier. so it might be that you don't need to take any action except for double checking this actually happens.

Yeah, just checked OptionsManager.UpdateRoutingManager(); queues segments to update in the next simulation step and then each  RecalculateSegment notifies listeners via OnSegmentNodesModified. All should work.


Going to version 4

We are going to embed new CHART.JS version 3.0.0.
This is forcing us to change a major release due to the amount of breaking changes that new version of CHART.JS is introducing.
There is a specific branch, named chartjs3.0.0 where all new implementations are coded, aligned with CHART.JS 3.0.0.
You can see the progress HERE.

Apart all breaking changes and new features, since version 4.0, chartjs/chartjs-plugin-annotation won't embed anymore because unmaintained.
Nevertheless Charba v4.0 will provide a very similar plugin, completely written in Java, where only few code changes will be needed in order to port the chartjs/chartjs-plugin-annotation in use to the new one.

In the current master, Datalabels plugin has been imported from a pepstock-org/chartjs-plugin-datalabels fork where the plugin has been changed in order to work with CHART.JS 3.
This is a temporary solution waiting for the upgrade of the original plugin by the community.

In the current master, INTL NumberFormat of ECMAScript Internationalization API has been mapped, into org.pepstock.charba.client.intl package in order to use when number formatting is needed and to implement the new format property for numeric ticks, available on ChartJS3.0.0-beta.
The new package is providing the full support on locale management (see CLocale and CLocaleBuilder, needed also to set the locale property to the chart options, available on ChartJS3.0.0-beta.

In the current master, Zoom plugin has been imported from a pepstock-org/chartjs-plugin-zoom fork where the plugin has been changed in order to work with CHART.JS 3.
This is a temporary solution waiting for the upgrade of the original plugin by the community.

CHART.JS 3 will drop the IE11 support and then Charba as well.

chartjs/chartjs-plugin-annotation won't embed anymore because unmaintained.
Nevertheless Charba v4.0 will provide a very similar plugin, completely written in Java, where only few code changes will be needed in order to port the chartjs/chartjs-plugin-annotation in use to the new one.

After a collaboration with the community which is managing the  chartjs/chartjs-plugin-annotation, the plugin has been embedded in Charba again.

chartjs/chartjs-plugin-annotation won't embed anymore because unmaintained.
Nevertheless Charba v4.0 will provide a very similar plugin, completely written in Java, where only few code changes will be needed in order to port the chartjs/chartjs-plugin-annotation in use to the new one.

After a collaboration with the community which is managing the  chartjs/chartjs-plugin-annotation, the plugin has been embedded in Charba again.


WIP: tenant dashboards

@ouranos just FYI, any comments?

FYI, waiting for https://github.com/maestrano/mno-enterprise/pull/535 to be approved & merged before adding specs for this.

Replace by: https://github.com/maestrano/mno-enterprise/pull/557


Erratum, Chapter 2, frame 15

I suppose this could be a pedagogical choice rather than an error per se, but the condition given in the English text is (I claim) not strong enough.
Specifically, it states that in order to make a tensor^{m+1}, we need to have a sequence of tensor^m's, where each one has the same number of elements. The actual text is
"Yes, but there's a condition.
All the tensors^m must have the same number of elements."
I claim that this condition is not strong enough: essentially, it's a "weak induction" condition when a "strong induction" condition is needed.
Specifically, consider this program:
#lang racket
(require malt)
(define t1 (tensor (tensor 3) (tensor 4)))
(define t2 (tensor (tensor 3 4) (tensor 4 5)))
(equal? (tlen t1) (tlen t2))
(tensor t1 t2)

The lengths of these two tensor^2s is both 2, but they cannot be combined into a tensor^3.
I'm not sure whether the solution here is to state a stronger condition, to explicitly defer the discussion until later, by saying something along the lines of "actually, the condition required is stronger but we're not going to talk about it now", or perhaps the text as written is a deliberate choice not to fully describe the condition.

... and I suppose everything is all cleaned up by chapter 2, frame 40, page 40, when the Rule of Uniform Shape is stated.


Adding climbing

Web slinging is great and fun until you hit a wall. Sometimes it would be nice to grab onto the wall or climb a little bit more and get on the roof or something.
There could be a button that you hold to climb or stick to the wall.
Maybe eventually basic locomotion would be a great fun addition. For moving across the roof.
Got my oculus quest today. I'm a professional js developer looking to branch into web VR.
I'm probably going to try adding this ability, unless someone is already working on it. Is there a better way to avoid duplicating efforts than a GitHub issue? Genuinely asking.

This.... Would be epic

Feel free, it will be very cool

Every time I start getting close to a wall, I'm always thinking to myself... How cool would it be to be able to land on a wall and run for a bit, then jump off and continue on doing whatever.
one of my favorite games is Titanfall, and it's because of that mobility where you can just keep going no matter what surface you're running on. It's like parkour heaven.
Spider-Man is all about parkour. 🤓
I like the idea of being able to press a button and either stick to a wall, or to run on the wall.

My Quest was supposed to arrive the day of my post, but was late and eventually got canceled. I have a new order arriving in 2 or 3 weeks. So a bit of a delay, but I see chrome and Firefox have WebXR emulator extensions. So I'll use that to get started testing some basic changes

Update when you get your headset plz I would really look forward to seeing some smart minds working on these types of things

how is it going?


Port to macOS (aka homebrew)

Since ucode runs fine on macOS it could be added to homebrew.
I ported it with this fun error message, however I'd keep this open as a reminder for myself to finish it in the near future
ucode:
  * GitHub repository not notable enough (<30 forks, <30 watchers and <75 stars)
Error: 1 problem in 1 formula detected

For reference the build definitions looks like this
class Ucode < Formula
  desc "Jinja-like micro templating"
  homepage "https://github.com/jow-/ucode"
  url "https://github.com/jow-/ucode/archive/refs/tags/v0.0.20220331.tar.gz"
  sha256 "7536e2454ff09092f32f508fc7a3dbe968292b81e88830bc37af1f3b728d7529"
  license "ISC"

  depends_on "cmake" => :build

  def install
     system "cmake", "-S", ".", "-B", "build",
       "-D", "NL80211_SUPPORT=OFF",
       "-D", "RTNL_SUPPORT=OFF",
       "-D", "UBUS_SUPPORT=OFF",
       "-D", "UCI_SUPPORT=OFF",
       "-D", "ULOOP_SUPPORT=OFF",
       *std_cmake_args
     system "cmake", "--build", "build"
  end

  test do
    environment_variables UCODE_BIN: working_dir/"build/ucode"
    environment_variables UCODE_LIB: working_dir/"build/"
    system ./tests/custom/run_tests.sh
  end
end


New eloquent collection loses item type information on creation

Larastan Version: v2.9.9
Laravel Version: v11.23.5

Description
Probably due to a generic model (SiteCheck<Checked>), creating an eloquent collection (ModelCollection) loses the information about the item type. Sometimes! If passed in as array, it works, but if passed in from a (non-eloquent) collection it doesn't. See type dumps:
Laravel code where the issue was found
// Not models:
\PHPStan\dumpType($checkables); // 🟩 array<int, App\Service\SiteChecker\Checkable>
// Create SiteCheck models from those non-models:
$siteChecks = collect($checkables)->map(function(Checkable $checkable) {
	return (new SiteCheck())->forceFill([
		'id' => $checkable->last_scid,
		'site_id' => $checkable->site_id,
		'check_id' => $checkable->check_id,
	]);
});

\PHPStan\dumpType($siteChecks); // 🟩 Illuminate\Support\Collection<int, App\Models\SiteCheck<App\Service\SiteChecker\Checked>>

// Now I want to eager-load a relationship in those models, so I need a ModelCollection instead of a Collection:

// Directly from the Collection loses information:
\PHPStan\dumpType(ModelCollection::make($siteChecks)); // 🟥 ModelCollection<(int|string), mixed>

// First to array keeps the type information, but makes a double ModelCollection!?:
\PHPStan\dumpType($siteChecks->all()); // 🟩 array<int, App\Models\SiteCheck<App\Service\SiteChecker\Checked>>
\PHPStan\dumpType(ModelCollection::make($siteChecks->all())); // 🟥  ModelCollection<int, ModelCollection<App\Service\SiteChecker\Checked>>

$checkables is an array of non-models, so I can't start with a ModelCollection, or Larastan will complain. I'm okay with that. First a collect(), then map to models, then make a ModelCollection, and load a relationship.
Creating the ModelCollection from the array from the Collection works, but creating it directly from the Collection does not. Very simple workaround, acceptable for now, but still a bug I think.
Once again, I THINK it's due to the genericness of SiteCheck. The second dump shows that genericness correctly: App\Models\SiteCheck<App\Service\SiteChecker\Checked>.

When converting a collection, try to use wrap instead of make:
\PHPStan\dumpType(ModelCollection::wrap($siteChecks));

When converting a collection, try to use wrap instead of make:

Still a double ModelCollection, in both methods (directly from Collection and through ->all() array).

My ModelCollection works everywhere else, but just in case:
use Illuminate\Database\Eloquent\Collection as EloquentCollection;

/**
 * @template TKey of array-key
 * @template TModel of Model
 *
 * @extends EloquentCollection<TKey, TModel>
 */
class ModelCollection extends EloquentCollection {

Model is in the same namespace, not the base eloquent Model, but it extends it.

Does it work with the base EloquentCollection?

It's not due to the genericness (wrong again!):
$contacts = Contact::all()->all();
\PHPStan\dumpType($contacts); // 🟩 array<int, App\Models\Contact>
$contacts = collect($contacts)->map(function(Contact $contact) {
	return (new Contact())->forceFill([
		'id' => $contact->id,
		'organization_id' => $contact->organization_id,
	]);
});
\PHPStan\dumpType($contacts); // 🟩 Illuminate\Support\Collection<int, Contact>
\PHPStan\dumpType(ModelCollection::make($contacts)); // 🟥 ModelCollection<(int|string), mixed>
\PHPStan\dumpType($contacts->all()); // 🟩 array<int, Contact>
\PHPStan\dumpType(ModelCollection::make($contacts->all())); // 🟩 ModelCollection<int, Contact> -- SURPRISINGLY WORKS!

That last dump surprised me. With ungeneric Contact it works, but with generic SiteCheck it doesn't. Maybe something to do with the genericness after all. Seems like 2 bugs.

What does \PHPStan\dumpType(ModelCollection::wrap($contacts));` do?

wrap() and make() have the same exact results.

Does it work with the base EloquentCollection?

No, same bug:
\PHPStan\dumpType($siteChecks); // 🟩 Illuminate\Support\Collection<int, App\Models\SiteCheck<App\Service\SiteChecker\Checked>>
\PHPStan\dumpType(EloquentCollection::make($siteChecks)); // 🟥 Illuminate\Database\Eloquent\Collection<(int|string), mixed>
\PHPStan\dumpType(EloquentCollection::make($siteChecks->all())); // 🟥 Illuminate\Database\Eloquent\Collection<int, Illuminate\Database\Eloquent\Collection<App\Service\SiteChecker\Checked>>

The double collection from the array is so weird. Why would it double wrap an array?

Yes, you can look into pipe / pipeInto

pipeInto has the same item type loss problem as expected:
\PHPStan\dumpType($siteChecks); // 🟩 array<int, Checkable> - non-models

$siteChecks = collect($checkables)
	->map(function(Checkable $checkable) {
		return (new SiteCheck())->forceFill([
			'id' => $checkable->last_scid,
			'site_id' => $checkable->site_id,
			'check_id' => $checkable->check_id,
		]);
	})
	->pipeInto(ModelCollection::class)
	->keyBy('site_id')
	->load('metrics');
\PHPStan\dumpType($siteChecks); // 🟥 ModelCollection<(int|string), App\Base\Model>

App\Base\Model instead of SiteCheck.
It seems that pipeInto loses even more information:
$contacts = Contact::all()->all();
\PHPStan\dumpType($contacts); // 🟩 array<int, App\Models\Contact>
$contacts = collect($contacts)
	->map(function(Contact $contact) {
		return (new Contact())->forceFill([
			'id' => $contact->id,
			'organization_id' => $contact->organization_id,
		]);
	})
	->pipeInto(ModelCollection::class);
\PHPStan\dumpType($contacts); // 🟥 App\Base\ModelCollection - not even a generic??

so that's never a good option.

Fixed with e19543da
Thank you!

Aah it was the EnumerableGenericStaticMethodDynamicStaticMethodReturnTypeExtension 😆 That has to be the longest class short name I've ever seen!
Does pipeInto() also work now? That would be very cool.

hmm. For pipeInto we might need a new extension. Or maybe it's solvable some other way. Can you open a feature request for that please? With minimal complete code example. Thank you.

But Collection::make() works? With $collection and with $collection->all()?

Yes it does.


Displaying generated output word-by-word instead of the whole output after inference.

As said in the title, I introduced TextStreamer (for inference.py) and TextIteratorStreamer (for demo.py) for word-by-word display of generated text. I'm not sure if it effects quality, but from my testing it does not.
I tested both inference.py and demo.py with this setup with the LongAlpaca-14B model.
I also removed the examples from the gradio demo as they were not included in the git repository, which induces an error when running demo.py.
Thank you for your hard work on this great project. I hope you continue to innovate and inspire, and I hope this is a good little contribution.

Hi,
Many thanks for your contribution. I have merged this pr into the main branch.
Regards,
Yukang Chen


Fluentd not pushing the logs to the AWS Elastic Search Service.

Hi,
I have a kubernetes cluster and I am trying to push the kubernetes logs to aws elastic search service.
Here is the link i am following and i could not able to push the logs to AWS Elastic Search.
https://github.com/heptio/fluentd-yaml
ubuntu@ip-172-31-12-81:~/test/fluentd-yaml$ kubectl logs  fluentd-kz4hp  --namespace=delete
2018-09-21 13:24:20 +0000 [info]: reading config file path="/fluentd/etc/fluent.conf"
2018-09-21 13:24:20 +0000 [info]: starting fluentd-0.12.32
2018-09-21 13:24:21 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '1.9.2'
2018-09-21 13:24:21 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '0.26.2'
2018-09-21 13:24:21 +0000 [info]: gem 'fluent-plugin-record-reformer' version '0.8.3'
2018-09-21 13:24:21 +0000 [info]: gem 'fluent-plugin-secure-forward' version '0.4.3'
2018-09-21 13:24:21 +0000 [info]: gem 'fluentd' version '0.12.32'
2018-09-21 13:24:21 +0000 [info]: adding match pattern="fluent." type="null"
2018-09-21 13:24:21 +0000 [info]: adding filter pattern="kubernetes." type="kubernetes_metadata"
2018-09-21 13:24:21 +0000 [info]: adding match pattern="es.**" type="aws-elasticsearch-service"
2018-09-21 13:24:21 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error="Unknown output plugin 'aws-elasticsearch-service'. Run 'gem search -rd fluent-plugin' to find plugins"
2018-09-21 13:24:21 +0000 [info]: process finished code=256
2018-09-21 13:24:21 +0000 [warn]: process died within 1 second. exit.
Here is the file I am using
fluentd-1.6.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: delete
  labels:
    k8s-app: fluentd-logging
    version: v1
    kubernetes.io/cluster-service: "true"
spec:
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
        version: v1
        kubernetes.io/cluster-service: "true"
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [{"key": "dedicated", "value": "master", "effect": "NoSchedule" }]
    spec:
      serviceAccountName: fluentd-service-account
      containers:
      - name: fluentd
        image: quay.io/fluent/fluentd-kubernetes-daemonset
        env:
          - name:  FLUENT_ELASTICSEARCH_HOST
            value: "https://search-fluentd-elasticsearch-mhksn7zojhmro732426eu7sk3a.ap-south-1.es.amazonaws.com"
          - name:  FLUENT_ELASTICSEARCH_PORT
            value: "443"
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: fluentdconf
          mountPath: /fluentd/etc/fluent.conf
          subPath: fluent.conf
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentdconf
        configMap:
          name: fluentd-configmap

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-configmap
  namespace: delete
data:
  fluent.conf: |-
    @include kubernetes.conf
    <match es.**>
       type "aws-elasticsearch-service"
       log_level info
       include_tag_key true
       host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
       port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
       logstash_format true
       buffer_chunk_limit 2M
       buffer_queue_limit 32
       flush_interval 5s
       max_retry_wait 30
       disable_retry_limit
       num_threads 8

       ### AWS ElasticSearch needs this set to false.  See
       ### https://discuss.elastic.co/t/elasitcsearch-ruby-raises-cannot-get-new-connection-from-pool-error/36252/10
       reload_connections false
    </match>

---
apiVersion: rbac.authorization.k8s.io/v1alpha1
kind: ClusterRoleBinding
metadata:
  name: fluentd-service-account
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: fluentd-service-account
subjects:
- kind: ServiceAccount
  name: fluentd-service-account
  namespace: delete

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: fluentd-service-account
  namespace: delete
rules:
  - apiGroups: ["*"]
    resources:
      - pods
      - namespaces
    verbs:
      - get
      - watch
      - list

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd-service-account
  namespace: delete

can any one help me how to push the logs from my kubernetes cluster to AWS Elastic Search.
Thanks in Advance.

As you can see from the logs
"config error file="/fluentd/etc/fluent.conf" error="Unknown output plugin 'aws-elasticsearch-service'. Run 'gem search -rd fluent-plugin' to find plugins"
you need to install the plugin. Look at https://github.com/atomita/fluent-plugin-aws-elasticsearch-service for more info.

Or instead, just
@type elasticsearch
instead of
type "aws-elasticsearch-service"

Hi dlbarduzzi,
Thanks for the prompt response.
While i create the pod using kubectl create -f working.yaml
I get the following error.
root@ip-172-31-31-68:~# create -f working.yaml
daemonset.extensions/fluentd created
configmap/fluentd-configmap created
serviceaccount/fluentd-service-account created
unable to recognize "working.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1alpha1"
unable to recognize "working.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1alpha1"
root@ip-172-31-31-68:~/home/ubuntu/test/fluentd-yaml# kubectl convert -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard-head.yaml --output-version=rbac.authorization.k8s.io/v1alpha1 | kubectl apply -f -
error: core.Secret is not suitable for converting to "rbac.authorization.k8s.io/v1alpha1"
error: no objects passed to apply
root@ip-172-31-31-68:~/home/ubuntu/test/fluentd-yaml#
I even tried with the above option, i am getting the following error. (@type elasticsearch)
root@ip-172-31-31-68:~/home/ubuntu/test/fluentd-yaml# kubectl logs fluentd-glpmk  --namespace=delete
2018-09-23 07:47:55 +0000 [info]: reading config file path="/fluentd/etc/fluent.conf"
2018-09-23 07:47:55 +0000 [info]: starting fluentd-0.12.32
2018-09-23 07:47:55 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '1.9.2'
2018-09-23 07:47:55 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '0.26.2'
2018-09-23 07:47:55 +0000 [info]: gem 'fluent-plugin-record-reformer' version '0.8.3'
2018-09-23 07:47:55 +0000 [info]: gem 'fluent-plugin-secure-forward' version '0.4.3'
2018-09-23 07:47:55 +0000 [info]: gem 'fluentd' version '0.12.32'
2018-09-23 07:47:55 +0000 [info]: adding match pattern="fluent." type="null"
2018-09-23 07:47:55 +0000 [info]: adding filter pattern="kubernetes." type="kubernetes_metadata"
2018-09-23 07:47:56 +0000 [info]: adding match pattern="es.**" type="elasticsearch"
2018-09-23 07:47:56 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error="Exception encountered fetching metadata from Kubernetes API endpoint: pods is forbidden: User "system:serviceaccount:delete:fluentd-service-account" cannot list pods at the cluster scope"
2018-09-23 07:47:56 +0000 [info]: process finished code=256
2018-09-23 07:47:56 +0000 [warn]: process died within 1 second. exit.
Thanks

Hi ravvreddy,
Can you share, How did you resolve this "Unknown output plugin 'aws-elasticsearch-service"

Currently, aws-elasticsearch-serivice image is not supported and provided in this repository.
I'm thinking about whether ought to support or not.

fluent-plugin-elasticseach cannot handle to push AWS Elasticsearch Service and this repository does not support debian-aws-elasticsearch-service image.
And this repository is pending to adding new image such as debian-aws-elasticsearch-service.
I decided that this should be closing.


compile.ml deserialize_from_blob uses mostly redundant specialization on extended boolean

This specialization on extended seems redundant, removing it might simplify things and share more code in binaries.
let deserialize_from_blob extended env ts =
    let ts_name = typ_seq_hash ts in
    let name =
      (* TODO(#3185): this specialization on `extended` seems redundant,
         removing it might simplify things *and* share more code in binaries.
         The only tricky bit might be the conditional Stack.dynamic_with_words bit... *)
      if extended
      then "@deserialize_extended<" ^ ts_name ^ ">"
      else "@deserialize<" ^ ts_name ^ ">" in
...

Don't fix until #3171 has merged


Hain v0.6.2 - hain.exe electron process persistent CPU spike

on my machine one of the hain threads is really consuming CPU usage.
(i5-2500k @ 4.5GHz, 16GB RAM)
It spikes from idle, to up to 50%, and will stick around there between 30-50% when doing nothing. this is weeks after indexing has completed.
Please let me know how to generate debug information on this so I can help to improve hain

By default, hain-plugin-filesearch re-indexes every 5 minutes I believe,
if you have some directories set to be recursive, this could be the cause.
In the same directory as hain.exe you'll find a hain-debug.log which
would show such activity.

wonder if this can be improved by storing last modified dates on recursive trees

I'd want to get confirmation that is the cause.  The best way to do this would be to run hain from a command shell (cmd.exe) which will show the output of what it's doing.  See what is being logged when the spike happens.  I haven't personally noticed any spikes while having it running.

Closing this as OP has not responded.

sorry, I forgot to update this issue. I am no longer seeing this issue in the latest versions. 0.6.4


updates for 0.9

Compiles without warnings on 0.9.0. Things still left to do in this PR once the core libs are properly released and pulp supports the new compiler flags:

[x] bump deps from RC's to "real" versions
[x] update devDependencies
[x] turn package.json scripts back into pulp commands

@spicydonuts

Awesome; and thanks for the heads up

rebased on top of master again and moved to 0.9.1 in devDependency. If we are willing to just use psc for the build and test commands we could get this merged right away.

@kRITZCREEK This looks great. Can you update the version of psc required in package.json?

^^^ npm install && pulp build should succeed. When that happens, package.json has been updated correctly (and obviously, pulp/psc have been deployed correctly).

@jdegoes I've updated psc to 0.9.1-rc.1. Right now pulp doesn't pass the correct flags to the compiler so this PR is just using psc directly behind npm run build. Changing that to be pulp again when pulp gets released is completely orthogonal to making purescript-aff 0.9 compatible which could unlock a bunch of reverse dependencies to release a 0.9 compatible version aswell.
Changing the Travis build process back to pulp, once it's updated, is hidden from anyone using purescript-aff as a library and could easily be put into the next bugfix release.

@kRITZCREEK This looks good. 👍 Anyone else have any comments? /cc @garyb @AppShipIt @spicydonuts


[flutter_tools] save size analysis files (created from --analyze-size) to .flutter-devtools folder

As of https://github.com/flutter/devtools/pull/2510, DevTools can now load files from the ~/.flutter-devtools/ directory. This is the only location from which DevTools can serve files for security reasons.
The --analyze-size command saves apk-analysis files to a default location. Can we change this to save the size analysis files to ~/.flutter-devtools/? This will be a key part of adding a quick link from command line that can launch DevTools with the generated size analysis file.
@jonahwilliams

Does it have to only be ~/.flutter-devtools, IIRC we do some specific logic to support XDG base directory specification for linux/macOS?
The output directory is configured here: https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/runner/flutter_command.dart#L803 , so that could be updated.

it just needs to be <user-home-directory>/.flutter-devtools/ however that is implemented. This is the path we specify in devtools: https://github.com/flutter/devtools/blob/master/packages/devtools_server/lib/src/file_system.dart#L18

Flutter tools should also be responsible for creating the <user-home-directory>/.flutter-devtools/ directory if it is not already created. This would be the case for a user who has never opened DevTools.

You might consider expanding that to include https://github.com/flutter/flutter/issues/59430 .


fix: grammar & spelling issues in README

Related issue

Scope of work

Fixed some spelling mistakes in README.md.
Found & fixed some grammatical mistakes also.

Please note that the path mentioned in the link in Line 40 of the README.md does not exist in the develop branch. This is a separate issue (#478) which should be dealt in a different PR, probably.
Screenshots of visual changes
Checklist

[ ] I followed composition rules for my component
[ ] I tested the component in most common device sizes (can be tested in Storybook from viewport addon in top menu)

Thank you, I'm a typo master so it's great that you found my mistakes and corrected them ;)


Hardcode the labels stable hash function instead of taking it as an option

This PR is a follow up of https://github.com/grafana/mimir-prometheus/pull/369. Instead of taking the sharding function as an option, we always use a "stable" hashing function which is guaranteed to not change over time.

As mentioned in Mimir's PR, we'd need to make sure that there's a test ensuring that we're using this specific hashing algorightm.

Yes, I'm wondering on it, but it's a set of tests we'll have in Mimir.

late to the party, but +1


Rename comic_cog.py to comic.py

Description

Renamed comic.py in cogs and respective changes made in main.py
Related Issue:
#199
closes # (issue)
Type of change

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have read the contributing guidelines
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation (if applicable)
[ ] My changes generate no new warnings

https://github.com/Vyvy-vi/TearDrops/blob/5a11525abbad37de9dda9f31fc54a4c4682b5656/bot/main.py#L30-L43
- 'cogs.comic_cog'
+ 'cogs.comic'

Thanks for opening this PR-!
Could you also make changes here?