0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

I am running into an issue with my Maven POM file where its unable to find spark dependency and is returning with error: Failed to read artifact descriptor for org.apache.spark:spark-streaming-kafka_2.10:jar:1.2.1

I've confirmed its not an issue with any corporate firewall as every other dependency is loaded in properly, just this one.

I have also been able to confirm in my maven settings it is attempting to pull from the following repo. I tried deleting the .m2 repo on my local machine in order to reload it, still no dice.

http://repo.maven.apache.org/maven2/org/apache/spark/spark-streaming-kafka_2.10/1.2.1/

Attached below is my pom file

<groupId>my.group.id</groupId>
<artifactId>sentiment</artifactId>
<version>1.0-SNAPSHOT</version>
<name>NPITWITTER</name>

<properties>
</properties>

    <dependencies>
        <dependency>
            <groupId>com.sparkjava</groupId>
            <artifactId>spark-core</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>


    </dependencies>

1 Answer

0 votes
by (31.4k points)

Once, while I was working with Intellij, I encountered the same issue.

I tried to fix the proxy settings, and also ran the command mvn –U clean install but none of them worked. 

But finally after a lot of research I got it resolved by enabling the setting given belowe (highlighted in yellow)

When you are on Intellij, go to File -> Settings -> Build, Execution, Deployments -> Maven Check the box, ‘Always update snapshots

Intellij Settings window snapshot

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...